Adaptive diffusion priors for accelerated MRI reconstruction

Joint coil sensitivity and motion correction in parallel MRI with a self-calibrating score-based diffusion model

Magnetic Resonance Imaging (MRI) stands as a powerful modality in clinical diagnosis. However, it faces challenges such as long acquisition time and vulnerability to motion-induced artifacts. While many existing motion correction algorithms have shown success, most fail to account for the impact of motion artifacts on coil sensitivity map (CSM) estimation during fast MRI reconstruction. This oversight can lead to significant performance degradation, as errors in the estimated CSMs can propagate and compromise motion correction. In this work, we propose JSMoCo, a novel method for jointly estimating motion parameters and time-varying coil sensitivity maps for under-sampled MRI reconstruction. The joint estimation presents a highly ill-posed inverse problem due to the increased number of unknowns. To address this challenge, we leverage score-based diffusion models as powerful priors and apply MRI physical principles to effectively constrain the solution space. Specifically, we parameterize rigid motion with trainable variables and model CSMs as polynomial functions. A Gibbs sampler is employed to ensure system consistency between the sensitivity maps and the reconstructed images, effectively preventing error propagation from pre-estimated sensitivity maps to the final reconstructed images. We evaluate JSMoCo through 2D and 3D motion correction experiments on simulated motion-corrupted fastMRI dataset and in-vivo real MRI brain scans. The results demonstrate that JSMoCo successfully reconstructs high-quality MRI images from under-sampled k-space data, achieving robust motion correction by accurately estimating time-varying coil sensitivities. The code is available at https://github.com/MeijiTian/JSMoCo.

Generative adversarial networks in medical image reconstruction: A systematic literature review

PURPOSE

Recent advancements in generative adversarial networks (GANs) have demonstrated substantial potential in medical image processing. Despite this progress, reconstructing images from incomplete data remains a challenge, impacting image quality. This systematic literature review explores the use of GANs in enhancing and reconstructing medical imaging data.

METHOD

A document survey of computing literature was conducted using the ACM Digital Library to identify relevant articles from journals and conference proceedings using keyword combinations, such as "generative adversarial networks or generative adversarial network," "medical image or medical imaging," and "image reconstruction."

RESULTS

Across the reviewed articles, there were 122 datasets used in 175 instances, 89 top metrics employed 335 times, 10 different tasks with a total count of 173, 31 distinct organs featured in 119 instances, and 18 modalities utilized in 121 instances, collectively depicting significant utilization of GANs in medical imaging. The adaptability and efficacy of GANs were showcased across diverse medical tasks, organs, and modalities, utilizing top public as well as private/synthetic datasets for disease diagnosis, including the identification of conditions like cancer in different anatomical regions. The study emphasized GAN's increasing integration and adaptability in diverse radiology modalities, showcasing their transformative impact on diagnostic techniques, including cross-modality tasks. The intricate interplay between network size, batch size, and loss function refinement significantly impacts GAN's performance, although challenges in training persist.

CONCLUSIONS

The study underscores GANs as dynamic tools shaping medical imaging, contributing significantly to image quality, training methodologies, and overall medical advancements, positioning them as substantial components driving medical advancements.

FDuDoCLNet: Fully dual-domain contrastive learning network for parallel MRI reconstruction

Magnetic resonance imaging (MRI) is a non-invasive medical imaging technique that is widely used for high-resolution imaging of soft tissues and organs. However, the slow speed of MRI imaging, especially in high-resolution or dynamic scans, makes MRI reconstruction an important research topic. Currently, MRI reconstruction methods based on deep learning (DL) have garnered significant attention, and they improve the reconstruction quality by learning complex image features. However, DL-based MR image reconstruction methods exhibit certain limitations. First, the existing reconstruction networks seldom account for the diverse frequency features in the wavelet domain. Second, existing dual-domain reconstruction methods may pay too much attention to the features of a single domain (such as the global information in the image domain or the local details in the wavelet domain), resulting in the loss of either critical global structures or fine details in certain regions of the reconstructed image. In this work, inspired by the lifting scheme in wavelet theory, we propose a novel Fully Dual-Domain Contrastive Learning Network (FDuDoCLNet) based on variational networks (VarNet) for accelerating PI in both the image and wavelet domains. It is composed of several cascaded dual-domain regularization units and data consistency (DC) layers, in which a novel dual-domain contrastive loss is introduced to optimize the reconstruction performance effectively. The proposed FDuDoCLNet was evaluated on the publicly available fastMRI multi-coil knee dataset under a 6× acceleration factor, achieving a PSNR of 34.439 dB and a SSIM of 0.895.

Ensemble and low-frequency mixing with diffusion models for accelerated MRI reconstruction

Magnetic resonance imaging (MRI) is an important imaging modality in medical diagnosis, providing comprehensive anatomical information with detailed tissue structures. However, the long scan time required to acquire high-quality MR images is a major challenge, especially in urgent clinical scenarios. Although diffusion models have achieved remarkable performance in accelerated MRI, there are several challenges. In particular, they struggle with the long inference time due to the high number of iterations in the reverse process of diffusion models. Additionally, they occasionally create artifacts or 'hallucinate' tissues that do not exist in the original anatomy. To address these problems, we propose ensemble and adaptive low-frequency mixing on the diffusion model, namely ELF-Diff for accelerated MRI. The proposed method consists of three key components in the reverse diffusion step: (1) optimization based on unified data consistency; (2) low-frequency mixing; and (3) aggregation of multiple perturbations of the predicted images for the ensemble in each step. We evaluate ELF-Diff on two MRI datasets, FastMRI and SKM-TEA. ELF-Diff surpasses other existing diffusion models for MRI reconstruction. Furthermore, extensive experiments, including a subtask of pathology detection, further demonstrate the superior anatomical precision of our method. ELF-Diff outperforms the existing state-of-the-art MRI reconstruction methods without being limited to specific undersampling patterns.

Illuminating the unseen: Advancing MRI domain generalization through causality

Deep learning methods have shown promise in accelerated MRI reconstruction but face significant challenges under domain shifts between training and testing datasets, such as changes in image contrasts, anatomical regions, and acquisition strategies. To address these challenges, we present the first domain generalization framework specifically designed for accelerated MRI reconstruction to robustness across unseen domains. The framework employs progressive strategies to enforce domain invariance, starting with image-level fidelity consistency to ensure robust reconstruction quality across domains, and feature alignment to capture domain-invariant representations. Advancing beyond these foundations, we propose a novel approach enforcing mechanism-level invariance, termed GenCA-MRI, which aligns intrinsic causal relationships within MRI data. We further develop a computational strategy that significantly reduces the complexity of causal alignment, ensuring its feasibility for real-world applications. Extensive experiments validate the framework's effectiveness, demonstrating both numerical and visual improvements over the baseline algorithm. GenCA-MRI presents the overall best performance, achieving a PSNR improvement up to 2.15 dB on fastMRI and 1.24 dB on IXI dataset at 8× acceleration, with superior performance in preserving anatomical details and mitigating domain-shift problem.

The state-of-the-art in cardiac MRI reconstruction: Results of the CMRxRecon challenge in MICCAI 2023

Cardiac magnetic resonance imaging (MRI) provides detailed and quantitative evaluation of the heart's structure, function, and tissue characteristics with high-resolution spatial-temporal imaging. However, its slow imaging speed and motion artifacts are notable limitations. Undersampling reconstruction, especially data-driven algorithms, has emerged as a promising solution to accelerate scans and enhance imaging performance using highly under-sampled data. Nevertheless, the scarcity of publicly available cardiac k-space datasets and evaluation platform hinder the development of data-driven reconstruction algorithms. To address this issue, we organized the Cardiac MRI Reconstruction Challenge (CMRxRecon) in 2023, in collaboration with the 26th International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI). CMRxRecon presented an extensive k-space dataset comprising cine and mapping raw data, accompanied by detailed annotations of cardiac anatomical structures. With overwhelming participation, the challenge attracted more than 285 teams and over 600 participants. Among them, 22 teams successfully submitted Docker containers for the testing phase, with 7 teams submitted for both cine and mapping tasks. All teams use deep learning based approaches, indicating that deep learning has predominately become a promising solution for the problem. The first-place winner of both tasks utilizes the E2E-VarNet architecture as backbones. In contrast, U-Net is still the most popular backbone for both multi-coil and single-coil reconstructions. This paper provides a comprehensive overview of the challenge design, presents a summary of the submitted results, reviews the employed methods, and offers an in-depth discussion that aims to inspire future advancements in cardiac MRI reconstruction models. The summary emphasizes the effective strategies observed in Cardiac MRI reconstruction, including backbone architecture, loss function, pre-processing techniques, physical modeling, and model complexity, thereby providing valuable insights for further developments in this field.

CAMP-Net: Consistency-Aware Multi-Prior Network for Accelerated MRI Reconstruction

Undersampling -space data in magnetic resonance imaging (MRI) reduces scan time but pose challenges in image reconstruction. Considerable progress has been made in reconstructing accelerated MRI. However, restoration of high-frequency image details in highly undersampled data remains challenging. To address this issue, we propose CAMP-Net, an unrolling-based Consistency-Aware Multi-Prior Network for accelerated MRI reconstruction. CAMP-Net leverages complementary multi-prior knowledge and multi-slice information from various domains to enhance reconstruction quality. Specifically, CAMP-Net comprises three interleaved modules for image enhancement, -space restoration, and calibration consistency, respectively. These modules jointly learn priors from data in image domain, -domain, and calibration region, respectively, in data-driven manner during each unrolled iteration. Notably, the encoded calibration prior knowledge extracted from auto-calibrating signals implicitly guides the learning of consistency-aware -space correlation for reliable interpolation of missing -space data. To maximize the benefits of image domain and -domain prior knowledge, the reconstructions are aggregated in a frequency fusion module, exploiting their complementary properties to optimize the trade-off between artifact removal and fine detail preservation. Additionally, we incorporate a surface data fidelity layer during the learning of -domain and calibration domain priors to prevent degradation of the reconstruction caused by padding-induced data imperfections. We evaluate the generalizability and robustness of our method on three large public datasets with varying acceleration factors and sampling patterns. The experimental results demonstrate that our method outperforms state-of-the-art approaches in terms of both reconstruction quality and mapping estimation, particularly in scenarios with high acceleration factors.

Self-supervised adversarial diffusion models for fast MRI reconstruction

BACKGROUND

Magnetic resonance imaging (MRI) offers excellent soft tissue contrast essential for diagnosis and treatment, but its long acquisition times can cause patient discomfort and motion artifacts.

PURPOSE

To propose a self-supervised deep learning-based compressed sensing MRI method named "Self-Supervised Adversarial Diffusion for MRI Accelerated Reconstruction (SSAD-MRI)" to accelerate data acquisition without requiring fully sampled datasets.

MATERIALS AND METHODS

We used the fastMRI multi-coil brain axialT 2 $\text{T}_{2}$ -weighted (T 2 $\text{T}_{2}$ -w) dataset from 1376 cases and single-coil brain quantitative magnetization prepared 2 rapid acquisition gradient echoesT 1 $\text{T}_{1}$ maps from 318 cases to train and test our model. Robustness against domain shift was evaluated using two out-of-distribution (OOD) datasets: multi-coil brain axial postcontrastT 1 $\text{T}_{1}$ -weighted (T 1 c $\text{T}_{1}\text{c}$ ) dataset from 50 cases and axial T1-weighted (T1-w) dataset from 50 patients. Data were retrospectively subsampled at acceleration ratesR ∈ { 2 × , 4 × , 8 × } $ R \in \lbrace 2\times, 4\times, 8\times \rbrace $ . SSAD-MRI partitions a random sampling pattern into two disjoint sets, ensuring data consistency during training. We compared our method with ReconFormer Transformer and SS-MRI, assessing performance using normalized mean squared error (NMSE), peak signal-to-noise ratio (PSNR), and structural similarity index (SSIM). Statistical tests included one-way analysis of variance and multi-comparison Tukey's honesty significant difference (HSD) tests.

RESULTS

SSAD-MRI preserved fine structures and brain abnormalities visually better than comparative methods atR = 8 × $ R=8\times$ for both multi-coil and single-coil datasets. It achieved the lowest NMSE atR ∈ { 4 × , 8 × } $ R \in \lbrace 4\times, 8\times \rbrace $ , and the highest PSNR and SSIM values at all acceleration rates for the multi-coil dataset. Similar trends were observed for the single-coil dataset, though SSIM values were comparable to ReconFormer atR ∈ { 2 × , 8 × } $ R \in \lbrace 2\times, 8\times \rbrace $ . These results were further confirmed by the voxel-wise correlation scatter plots. OOD results showed significant (p≪ 10 - 5 $ \ll 10^{-5}$ ) improvements in undersampled image quality after reconstruction.

CONCLUSIONS

SSAD-MRI successfully reconstructs fully sampled images without utilizing them in the training step, potentially reducing imaging costs and enhancing image quality crucial for diagnosis and treatment.

SPIRiT-Diffusion: Self-Consistency Driven Diffusion Model for Accelerated MRI

Diffusion models have emerged as a leading methodology for image generation and have proven successful in the realm of magnetic resonance imaging (MRI) reconstruction. However, existing reconstruction methods based on diffusion models are primarily formulated in the image domain, making the reconstruction quality susceptible to inaccuracies in coil sensitivity maps (CSMs). k-space interpolation methods can effectively address this issue but conventional diffusion models are not readily applicable in k-space interpolation. To overcome this challenge, we introduce a novel approach called SPIRiT-Diffusion, which is a diffusion model for k-space interpolation inspired by the iterative self-consistent SPIRiT method. Specifically, we utilize the iterative solver of the self-consistent term (i.e., k-space physical prior) in SPIRiT to formulate a novel stochastic differential equation (SDE) governing the diffusion process. Subsequently, k-space data can be interpolated by executing the diffusion process. This innovative approach highlights the optimization model's role in designing the SDE in diffusion models, enabling the diffusion process to align closely with the physics inherent in the optimization model-a concept referred to as model-driven diffusion. We evaluated the proposed SPIRiT-Diffusion method using a 3D joint intracranial and carotid vessel wall imaging dataset. The results convincingly demonstrate its superiority over image-domain reconstruction methods, achieving high reconstruction quality even at a substantial acceleration rate of 10. Our code are available at https://github.com/zhyjSIAT/SPIRiT-Diffusion.

Highly accelerated MRI via implicit neural representation guided posterior sampling of diffusion models

Reconstructing high-fidelity magnetic resonance (MR) images from under-sampled k-space is a commonly used strategy to reduce scan time. The posterior sampling of diffusion models based on the real measurement data holds significant promise of improved reconstruction accuracy. However, traditional posterior sampling methods often lack effective data consistency guidance, leading to inaccurate and unstable reconstructions. Implicit neural representation (INR) has emerged as a powerful paradigm for solving inverse problems by modeling a signal's attributes as a continuous function of spatial coordinates. In this study, we present a novel posterior sampler for diffusion models using INR, named DiffINR. The INR-based component incorporates both the diffusion prior distribution and the MRI physical model to ensure high data fidelity. DiffINR demonstrates superior performance on in-distribution datasets with remarkable accuracy, even under high acceleration factors (up to R = 12 in single-channel reconstruction). Furthermore, DiffINR exhibits excellent generalizability across various tissue contrasts and anatomical structures with low uncertainty. Overall, DiffINR significantly improves MRI reconstruction in terms of accuracy, generalizability and stability, paving the way for further accelerating MRI acquisition. Notably, our proposed framework can be a generalizable framework to solve inverse problems in other medical imaging tasks.

Data- and Physics-Driven Deep Learning Based Reconstruction for Fast MRI: Fundamentals and Methodologies

Magnetic Resonance Imaging (MRI) is a pivotal clinical diagnostic tool, yet its extended scanning times often compromise patient comfort and image quality, especially in volumetric, temporal and quantitative scans. This review elucidates recent advances in MRI acceleration via data and physics-driven models, leveraging techniques from algorithm unrolling models, enhancement-based methods, and plug-and-play models to the emerging full spectrum of generative model-based methods. We also explore the synergistic integration of data models with physics-based insights, encompassing the advancements in multi-coil hardware accelerations like parallel imaging and simultaneous multi-slice imaging, and the optimization of sampling patterns. We then focus on domain-specific challenges and opportunities, including image redundancy exploitation, image integrity, evaluation metrics, data heterogeneity, and model generalization. This work also discusses potential solutions and future research directions, with an emphasis on the role of data harmonization and federated learning for further improving the general applicability and performance of these methods in MRI reconstruction.

ACGRHA-Net: Accelerated multi-contrast MR imaging with adjacency complementary graph assisted residual hybrid attention network

Multi-contrast magnetic resonance (MR) imaging is an advanced technology used in medical diagnosis, but the long acquisition process can lead to patient discomfort and limit its broader application. Shortening acquisition time by undersampling k-space data introduces noticeable aliasing artifacts. To address this, we propose a method that reconstructs multi-contrast MR images from zero-filled data by utilizing a fully-sampled auxiliary contrast MR image as a prior to learn an adjacency complementary graph. This graph is then combined with a residual hybrid attention network, forming the adjacency complementary graph assisted residual hybrid attention network (ACGRHA-Net) for multi-contrast MR image reconstruction. Specifically, the optimal structural similarity is represented by a graph learned from the fully sampled auxiliary image, where the node features and adjacency matrices are designed to precisely capture structural information among different contrast images. This structural similarity enables effective fusion with the target image, improving the detail reconstruction. Additionally, a residual hybrid attention module is designed in parallel with the graph convolution network, allowing it to effectively capture key features and adaptively emphasize these important features in target contrast MR images. This strategy prioritizes crucial information while preserving shallow features, thereby achieving comprehensive feature fusion at deeper levels to enhance multi-contrast MR image reconstruction. Extensive experiments on the different datasets, using various sampling patterns and accelerated factors demonstrate that the proposed method outperforms the current state-of-the-art reconstruction methods.

Improving the lesion appearance on FLAIR images synthetized from quantitative MRI: a fast, hybrid approach

OBJECTIVE

The image quality of synthetized FLAIR (fluid attenuated inversion recovery) images is generally inferior to its conventional counterpart, especially regarding the lesion contrast mismatch. This work aimed to improve the lesion appearance through a hybrid methodology.

MATERIALS AND METHODS

We combined a full brain 5-min MR-STAT acquisition followed by FLAIR synthetization step with an ultra-under sampled conventional FLAIR sequence and performed the retrospective and prospective analysis of the proposed method on the patient datasets and a healthy volunteer.

RESULTS

All performance metrics of the proposed hybrid FLAIR images on patient datasets were significantly higher than those of the physics-based FLAIR images (p < 0.005), and comparable to those of conventional FLAIR images. The small difference between prospective and retrospective analysis on a healthy volunteer demonstrated the validity of the retrospective analysis of the hybrid method as presented for the patient datasets.

DISCUSSION

The proposed hybrid FLAIR achieved an improved lesion appearance in the clinical cases with neurological diseases compared to the physics-based FLAIR images, Future prospective work on patient data will address the validation of the method from a diagnostic perspective by radiological inspection of the new images over a larger patient cohort.

A review of deep learning-based reconstruction methods for accelerated MRI using spatiotemporal and multi-contrast redundancies

Accelerated magnetic resonance imaging (MRI) has played an essential role in reducing data acquisition time for MRI. Acceleration can be achieved by acquiring fewer data points in k-space, which results in various artifacts in the image domain. Conventional reconstruction methods have resolved the artifacts by utilizing multi-coil information, but with limited robustness. Recently, numerous deep learning-based reconstruction methods have been developed, enabling outstanding reconstruction performances with higher acceleration. Advances in hardware and developments of specialized network architectures have produced such achievements. Besides, MRI signals contain various redundant information including multi-coil redundancy, multi-contrast redundancy, and spatiotemporal redundancy. Utilization of the redundant information combined with deep learning approaches allow not only higher acceleration, but also well-preserved details in the reconstructed images. Consequently, this review paper introduces the basic concepts of deep learning and conventional accelerated MRI reconstruction methods, followed by review of recent deep learning-based reconstruction methods that exploit various redundancies. Lastly, the paper concludes by discussing the challenges, limitations, and potential directions of future developments.

FEFA: Frequency Enhanced Multi-Modal MRI Reconstruction With Deep Feature Alignment

Integrating complementary information from multiple magnetic resonance imaging (MRI) modalities is often necessary to make accurate and reliable diagnostic decisions. However, the different acquisition speeds of these modalities mean that obtaining information can be time consuming and require significant effort. Reference-based MRI reconstruction aims to accelerate slower, under-sampled imaging modalities, such as T2-modality, by utilizing redundant information from faster, fully sampled modalities, such as T1-modality. Unfortunately, spatial misalignment between different modalities often negatively impacts the final results. To address this issue, we propose FEFA, which consists of cascading FEFA blocks. The FEFA block first aligns and fuses the two modalities at the feature level. The combined features are then filtered in the frequency domain to enhance the important features while simultaneously suppressing the less essential ones, thereby ensuring accurate reconstruction. Furthermore, we emphasize the advantages of combining the reconstruction results from multiple cascaded blocks, which also contributes to stabilizing the training process. Compared to existing registration-then-reconstruction and cross-attention-based approaches, our method is end-to-end trainable without requiring additional supervision, extensive parameters, or heavy computation. Experiments on the public fastMRI, IXI and in-house datasets demonstrate that our approach is effective across various under-sampling patterns and ratios.

Compressed SVD-based L + S model to reconstruct undersampled dynamic MRI data using parallel architecture

BACKGROUND

Magnetic Resonance Imaging (MRI) is a highly demanded medical imaging system due to high resolution, large volumetric coverage, and ability to capture the dynamic and functional information of body organs e.g. cardiac MRI is employed to assess cardiac structure and evaluate blood flow dynamics through the cardiac valves. Long scan time is the main drawback of MRI, which makes it difficult for the patients to remain still during the scanning process.

OBJECTIVE

By collecting fewer measurements, MRI scan time can be shortened, but this undersampling causes aliasing artifacts in the reconstructed images. Advanced image reconstruction algorithms have been used in literature to overcome these undersampling artifacts. These algorithms are computationally expensive and require a long time for reconstruction which makes them infeasible for real-time clinical applications e.g. cardiac MRI. However, exploiting the inherent parallelism in these algorithms can help to reduce their computation time.

METHODS

Low-rank plus sparse (L+S) matrix decomposition model is a technique used in literature to reconstruct the highly undersampled dynamic MRI (dMRI) data at the expense of long reconstruction time. In this paper, Compressed Singular Value Decomposition (cSVD) model is used in L+S decomposition model (instead of conventional SVD) to reduce the reconstruction time. The results provide improved quality of the reconstructed images. Furthermore, it has been observed that cSVD and other parts of the L+S model possess highly parallel operations; therefore, a customized GPU based parallel architecture of the modified L+S model has been presented to further reduce the reconstruction time.

RESULTS

Four cardiac MRI datasets (three different cardiac perfusion acquired from different patients and one cardiac cine data), each with different acceleration factors of 2, 6 and 8 are used for experiments in this paper. Experimental results demonstrate that using the proposed parallel architecture for the reconstruction of cardiac perfusion data provides a speed-up factor up to 19.15× (with memory latency) and 70.55× (without memory latency) in comparison to the conventional CPU reconstruction with no compromise on image quality.

CONCLUSION

The proposed method is well-suited for real-time clinical applications, offering a substantial reduction in reconstruction time.

Correlated and Multi-Frequency Diffusion Modeling for Highly Under-Sampled MRI Reconstruction

Given the obstacle in accentuating the reconstruction accuracy for diagnostically significant tissues, most existing MRI reconstruction methods perform targeted reconstruction of the entire MR image without considering fine details, especially when dealing with highly under-sampled images. Therefore, a considerable volume of efforts has been directed towards surmounting this challenge, as evidenced by the emergence of numerous methods dedicated to preserving high-frequency content as well as fine textural details in the reconstructed image. In this case, exploring the merits associated with each method of mining high-frequency information and formulating a reasonable principle to maximize the joint utilization of these approaches will be a more effective solution to achieve accurate reconstruction. Specifically, this work constructs an innovative principle named Correlated and Multi-frequency Diffusion Model (CM-DM) for highly under-sampled MRI reconstruction. In essence, the rationale underlying the establishment of such principle lies not in assembling arbitrary models, but in pursuing the effective combinations and replacement of components. It also means that the novel principle focuses on forming a correlated and multi-frequency prior through different high-frequency operators in the diffusion process. Moreover, multi-frequency prior further constraints the noise term closer to the target distribution in the frequency domain, thereby making the diffusion process converge faster. Experimental results verify that the proposed method achieved superior reconstruction accuracy, with a notable enhancement of approximately 2dB in PSNR compared to state-of-the-art methods.

A Plug-In Graph Neural Network to Boost Temporal Sensitivity in fMRI Analysis

Learning-based methods offer performance leaps over traditional methods in classification analysis of high-dimensional functional MRI (fMRI) data. In this domain, deep-learning models that analyze functional connectivity (FC) features among brain regions have been particularly promising. However, many existing models receive as input temporally static FC features that summarize inter-regional interactions across an entire scan, reducing the temporal sensitivity of classifiers by limiting their ability to leverage information on dynamic FC features of brain activity. To improve the performance of baseline classification models without compromising efficiency, here we propose a novel plug-in based on a graph neural network, GraphCorr, to provide enhanced input features to baseline models. The proposed plug-in computes a set of latent FC features with enhanced temporal information while maintaining comparable dimensionality to static features. Taking brain regions as nodes and blood-oxygen-level-dependent (BOLD) signals as node inputs, GraphCorr leverages a node embedder module based on a transformer encoder to capture dynamic latent representations of BOLD signals. GraphCorr also leverages a lag filter module to account for delayed interactions across nodes by learning correlational features of windowed BOLD signals across time delays. These two feature groups are then fused via a message passing algorithm executed on the formulated graph. Comprehensive demonstrations on three public datasets indicate improved classification performance for several state-of-the-art graph and convolutional baseline models when they are augmented with GraphCorr.

Enhancing quality and speed in database-free neural network reconstructions of undersampled MRI with SCAMPI

PURPOSE

We present SCAMPI (Sparsity Constrained Application of deep Magnetic resonance Priors for Image reconstruction), an untrained deep Neural Network for MRI reconstruction without previous training on datasets. It expands the Deep Image Prior approach with a multidomain, sparsity-enforcing loss function to achieve higher image quality at a faster convergence speed than previously reported methods.

METHODS

Two-dimensional MRI data from the FastMRI dataset with Cartesian undersampling in phase-encoding direction were reconstructed for different acceleration rates for single coil and multicoil data.

RESULTS

The performance of our architecture was compared to state-of-the-art Compressed Sensing methods and ConvDecoder, another untrained Neural Network for two-dimensional MRI reconstruction. SCAMPI outperforms these by better reducing undersampling artifacts and yielding lower error metrics in multicoil imaging. In comparison to ConvDecoder, the U-Net architecture combined with an elaborated loss-function allows for much faster convergence at higher image quality. SCAMPI can reconstruct multicoil data without explicit knowledge of coil sensitivity profiles. Moreover, it is a novel tool for reconstructing undersampled single coil k-space data.

CONCLUSION

Our approach avoids overfitting to dataset features, that can occur in Neural Networks trained on databases, because the network parameters are tuned only on the reconstruction data. It allows better results and faster reconstruction than the baseline untrained Neural Network approach.

Super-resolution reconstruction for early cervical cancer magnetic resonance imaging based on deep learning

This study aims to develop a super-resolution (SR) algorithm tailored specifically for enhancing the image quality and resolution of early cervical cancer (CC) magnetic resonance imaging (MRI) images. The proposed method is subjected to both qualitative and quantitative analyses, thoroughly investigating its performance across various upscaling factors and assessing its impact on medical image segmentation tasks. The innovative SR algorithm employed for reconstructing early CC MRI images integrates complex architectures and deep convolutional kernels. Training is conducted on matched pairs of input images through a multi-input model. The research findings highlight the significant advantages of the proposed SR method on two distinct datasets at different upscaling factors. Specifically, at a 2× upscaling factor, the sagittal test set outperforms the state-of-the-art methods in the PSNR index evaluation, second only to the hybrid attention transformer, while the axial test set outperforms the state-of-the-art methods in both PSNR and SSIM index evaluation. At a 4× upscaling factor, both the sagittal test set and the axial test set achieve the best results in the evaluation of PNSR and SSIM indicators. This method not only effectively enhances image quality, but also exhibits superior performance in medical segmentation tasks, thereby providing a more reliable foundation for clinical diagnosis and image analysis.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Parallel diffusion models promote high detail-fidelity photoacoustic microscopy in sparse sampling

Reconstructing sparsely sampled data is fundamental for achieving high spatiotemporal resolution photoacoustic microscopy (PAM) of microvascular morphology in vivo. Convolutional networks (CNN) and generative adversarial networks (GAN) have been introduced to high-speed PAM, but due to the use of upsampling in CNN-based networks to restore details and the instability in GAN training, they struggle to learn the entangled microvascular network structure and vascular texture features, resulting in only achieving low detail-fidelity imaging of microvascular. The diffusion models is richly sampled and can generate high-quality images, which is very helpful for the complex vascular features in PAM. Here, we propose an approach named parallel diffusion models (PDM) with parallel learning of Noise task and Image task, where the Noise task optimizes through variational lower bounds to generate microvascular structures that are visually realistic, and the Image task improves the fidelity of the generated microvascular details through image-based loss. With only 1.56% of fully sampled pixels from photoacoustic human oral data, PDM achieves an LPIPS of 0.199. Additionally, using PDM in high-speed 16x PAM prevents breathing artifacts and image distortion issues caused by low-speed sampling, reduces the standard deviation of the Row-wise Self-Correlation Coefficient, and maintains high image quality. It achieves high confidence in reconstructing detailed information from sparsely sampled data and will promote the application of reconstructed sparsely sampled data in realizing high spatiotemporal resolution PAM.

Learning to deep learning: statistics and a paradigm test in selecting a UNet architecture to enhance MRI

OBJECTIVE

This study aims to assess the statistical significance of training parameters in 240 dense UNets (DUNets) used for enhancing low Signal-to-Noise Ratio (SNR) and undersampled MRI in various acquisition protocols. The objective is to determine the validity of differences between different DUNet configurations and their impact on image quality metrics.

MATERIALS AND METHODS

To achieve this, we trained all DUNets using the same learning rate and number of epochs, with variations in 5 acquisition protocols, 24 loss function weightings, and 2 ground truths. We calculated evaluation metrics for two metric regions of interest (ROI). We employed both Analysis of Variance (ANOVA) and Mixed Effects Model (MEM) to assess the statistical significance of the independent parameters, aiming to compare their efficacy in revealing differences and interactions among fixed parameters.

RESULTS

ANOVA analysis showed that, except for the acquisition protocol, fixed variables were statistically insignificant. In contrast, MEM analysis revealed that all fixed parameters and their interactions held statistical significance. This emphasizes the need for advanced statistical analysis in comparative studies, where MEM can uncover finer distinctions often overlooked by ANOVA.

DISCUSSION

These findings highlight the importance of utilizing appropriate statistical analysis when comparing different deep learning models. Additionally, the surprising effectiveness of the UNet architecture in enhancing various acquisition protocols underscores the potential for developing improved methods for characterizing and training deep learning models. This study serves as a stepping stone toward enhancing the transparency and comparability of deep learning techniques for medical imaging applications.

Dual-domain faster Fourier convolution based network for MR image reconstruction

Deep learning methods for fast MRI have shown promise in reconstructing high-quality images from undersampled multi-coil k-space data, leading to reduced scan duration. However, existing methods encounter challenges related to limited receptive fields in dual-domain (k-space and image domains) reconstruction networks, rigid data consistency operations, and suboptimal refinement structures, which collectively restrict overall reconstruction performance. This study introduces a comprehensive framework that addresses these challenges and enhances MR image reconstruction quality. Firstly, we propose Faster Inverse Fourier Convolution (FasterIFC), a frequency domain convolutional operator that significantly expands the receptive field of k-space domain reconstruction networks. Expanding the information extraction range to the entire frequency spectrum according to the spectral convolution theorem in Fourier theory enables the network to easily utilize richer redundant long-range information from adjacent, symmetrical, and diagonal locations of multi-coil k-space data. Secondly, we introduce a novel softer Data Consistency (softerDC) layer, which achieves an enhanced balance between data consistency and smoothness. This layer facilitates the implementation of diverse data consistency strategies across distinct frequency positions, addressing the inflexibility observed in current methods. Finally, we present the Dual-Domain Faster Fourier Convolution Based Network (D2F2), which features a centrosymmetric dual-domain parallel structure based on FasterIFC. This architecture optimally leverages dual-domain data characteristics while substantially expanding the receptive field in both domains. Coupled with the softerDC layer, D2F2 demonstrates superior performance on the NYU fastMRI dataset at multiple acceleration factors, surpassing state-of-the-art methods in both quantitative and qualitative evaluations.

Deep learning for accelerated and robust MRI reconstruction

Deep learning (DL) has recently emerged as a pivotal technology for enhancing magnetic resonance imaging (MRI), a critical tool in diagnostic radiology. This review paper provides a comprehensive overview of recent advances in DL for MRI reconstruction, and focuses on various DL approaches and architectures designed to improve image quality, accelerate scans, and address data-related challenges. It explores end-to-end neural networks, pre-trained and generative models, and self-supervised methods, and highlights their contributions to overcoming traditional MRI limitations. It also discusses the role of DL in optimizing acquisition protocols, enhancing robustness against distribution shifts, and tackling biases. Drawing on the extensive literature and practical insights, it outlines current successes, limitations, and future directions for leveraging DL in MRI reconstruction, while emphasizing the potential of DL to significantly impact clinical imaging practices.

Deep learning-based super-resolution of structural brain MRI at 1.5 T: application to quantitative volume measurement

OBJECTIVE

This study investigated the feasibility of using deep learning-based super-resolution (DL-SR) technique on low-resolution (LR) images to generate high-resolution (HR) MR images with the aim of scan time reduction. The efficacy of DL-SR was also assessed through the application of brain volume measurement (BVM).

MATERIALS AND METHODS

In vivo brain images acquired with 3D-T1W from various MRI scanners were utilized. For model training, LR images were generated by downsampling the original 1 mm-2 mm isotropic resolution images. Pairs of LR and HR images were used for training 3D residual dense net (RDN). For model testing, actual scanned 2 mm isotropic resolution 3D-T1W images with one-minute scan time were used. Normalized root-mean-square error (NRMSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) were used for model evaluation. The evaluation also included brain volume measurement, with assessments of subcortical brain regions.

RESULTS

The results showed that DL-SR model improved the quality of LR images compared with cubic interpolation, as indicated by NRMSE (24.22% vs 30.13%), PSNR (26.19 vs 24.65), and SSIM (0.96 vs 0.95). For volumetric assessments, there were no significant differences between DL-SR and actual HR images (p > 0.05, Pearson's correlation > 0.90) at seven subcortical regions.

DISCUSSION

The combination of LR MRI and DL-SR enables addressing prolonged scan time in 3D MRI scans while providing sufficient image quality without affecting brain volume measurement.

High-Dimensional MR Reconstruction Integrating Subspace and Adaptive Generative Models

OBJECTIVE

To develop a new method that integrates subspace and generative image models for high-dimensional MR image reconstruction.

METHODS

We proposed a formulation that synergizes a low-dimensional subspace model of high-dimensional images, an adaptive generative image prior serving as spatial constraints on the sequence of "contrast-weighted" images or spatial coefficients of the subspace model, and a conventional sparsity regularization. A special pretraining plus subject-specific network adaptation strategy was proposed to construct an accurate generative-network-based representation for images with varying contrasts. An iterative algorithm was introduced to jointly update the subspace coefficients and the multi-resolution latent space of the generative image model that leveraged an recently proposed intermediate layer optimization technique for network inversion.

RESULTS

We evaluated the utility of the proposed method for two high-dimensional imaging applications: accelerated MR parameter mapping and high-resolution MR spectroscopic imaging. Improved performance over state-of-the-art subspace-based methods was demonstrated in both cases.

CONCLUSION

The proposed method provided a new way to address high-dimensional MR image reconstruction problems by incorporating an adaptive generative model as a data-driven spatial prior for constraining subspace reconstruction.

SIGNIFICANCE

Our work demonstrated the potential of integrating data-driven and adaptive generative priors with canonical low-dimensional modeling for high-dimensional imaging problems.

Radial Undersampled MRI Reconstruction Using Deep Learning With Mutual Constraints Between Real and Imaginary Components of K-Space

The deep learning method is an efficient solution for improving the quality of undersampled magnetic resonance (MR) image reconstruction while reducing lengthy data acquisition. Most deep learning methods neglect the mutual constraints between the real and imaginary components of complex-valued k-space data. In this paper, a new complex-valued convolutional neural network, namely, Dense-U-Dense Net (DUD-Net), is proposed to interpolate the undersampled k-space data and reconstruct MR images. The proposed network comprises dense layers, U-Net, and other dense layers in sequence. The dense layers are used to simulate the mutual constraints between real and imaginary components, and U-Net performs feature sparsity and interpolation estimation for k-space data. Two MRI datasets were used to evaluate the proposed method: brain magnitude-only MR images and knee complex-valued k-space data. Several operations were conducted for data preprocessing. First, the complex-valued MR images were synthesized by phase modulation on magnitude-only images. Second, a radial trajectory based on the golden angle was used for k-space undersampling, whereby a reversible normalization method was proposed to balance the distribution of positive and negative values in k-space data. The optimal performance of DUD-Net was demonstrated based on a quantitative evaluation of inter-method and intra-method comparisons. When compared with other methods, significant improvements were achieved, PSNRs were increased by 10.78 and 5.74dB, whereas RMSEs were decreased by 71.53% and 30.31% for magnitude and phase image, respectively. It is concluded that DUD-Net significantly improves the performance of MR image reconstruction.

DiffGAN: An adversarial diffusion model with local transformer for MRI reconstruction

Magnetic resonance imaging (MRI) is non-invasive and crucial for clinical diagnosis, but it has long acquisition time and aliasing artifacts. Accelerated imaging techniques can effectively reduce the scanning time of MRI, thereby decreasing the anxiety and discomfort of patients. Vision Transformer (ViT) based methods have greatly improved MRI image reconstruction, but their computational complexity and memory requirements for the self-attention mechanism grow quadratically with image resolution, which limits their use for high resolution images. In addition, the current generative adversarial networks in MRI reconstruction are difficult to train stably. To address these problems, we propose a Local Vision Transformer (LVT) based adversarial Diffusion model (Diff-GAN) for accelerating MRI reconstruction. We employ a generative adversarial network (GAN) as the reverse diffusion model to enable large diffusion steps. In the forward diffusion module, we use a diffusion process to generate Gaussian mixture distribution noise, which mitigates the gradient vanishing issue in GAN training. This network leverages the LVT module with the local self-attention, which can capture high-quality local features and detailed information. We evaluate our method on four datasets: IXI, MICCAI 2013, MRNet and FastMRI, and demonstrate that Diff-GAN can outperform several state-of-the-art GAN-based methods for MRI reconstruction.

A cross-domain complex convolution neural network for undersampled magnetic resonance image reconstruction

To introduce a new cross-domain complex convolution neural network for accurate MR image reconstruction from undersampled k-space data. Most reconstruction methods utilize neural networks or cascade neural networks in either the image domain and/or the k-space domain. However, these methods encounter several challenges: 1) Applying neural networks directly in the k-space domain is suboptimal for feature extraction; 2) Classic image-domain networks have difficulty in fully extracting texture features; and 3) Existing cross-domain methods still face challenges in extracting and fusing features from both image and k-space domains simultaneously. In this work, we propose a novel deep-learning-based 2-D single-coil complex-valued MR reconstruction network termed TEID-Net. TEID-Net integrates three modules: 1) TE-Net, an image-domain-based sub-network designed to enhance contrast in input features by incorporating a Texture Enhancement Module; 2) ID-Net, an intermediate-domain sub-network tailored to operate in the image-Fourier space, with the specific goal of reducing aliasing artifacts realized by leveraging the superior incoherence property of the decoupled one-dimensional signals; and 3) TEID-Net, a cross-domain reconstruction network in which ID-Nets and TE-Nets are combined and cascaded to boost the quality of image reconstruction further. Extensive experiments have been conducted on the fastMRI and Calgary-Campinas datasets. Results demonstrate the effectiveness of the proposed TEID-Net in mitigating undersampling-induced artifacts and producing high-quality image reconstructions, outperforming several state-of-the-art methods while utilizing fewer network parameters. The cross-domain TEID-Net excels in restoring tissue structures and intricate texture details. The results illustrate that TEID-Net is particularly well-suited for regular Cartesian undersampling scenarios.

One model to unite them all: Personalized federated learning of multi-contrast MRI synthesis

Curation of large, diverse MRI datasets via multi-institutional collaborations can help improve learning of generalizable synthesis models that reliably translate source- onto target-contrast images. To facilitate collaborations, federated learning (FL) adopts decentralized model training while mitigating privacy concerns by avoiding sharing of imaging data. However, conventional FL methods can be impaired by the inherent heterogeneity in the data distribution, with domain shifts evident within and across imaging sites. Here we introduce the first personalized FL method for MRI Synthesis (pFLSynth) that improves reliability against data heterogeneity via model specialization to individual sites and synthesis tasks (i.e., source-target contrasts). To do this, pFLSynth leverages an adversarial model equipped with novel personalization blocks that control the statistics of generated feature maps across the spatial/channel dimensions, given latent variables specific to sites and tasks. To further promote communication efficiency and site specialization, partial network aggregation is employed over later generator stages while earlier generator stages and the discriminator are trained locally. As such, pFLSynth enables multi-task training of multi-site synthesis models with high generalization performance across sites and tasks. Comprehensive experiments demonstrate the superior performance and reliability of pFLSynth in MRI synthesis against prior federated methods.

Image Reconstruction for Accelerated MR Scan With Faster Fourier Convolutional Neural Networks

High quality image reconstruction from undersampled k -space data is key to accelerating MR scanning. Current deep learning methods are limited by the small receptive fields in reconstruction networks, which restrict the exploitation of long-range information, and impede the mitigation of full-image artifacts, particularly in 3D reconstruction tasks. Additionally, the substantial computational demands of 3D reconstruction considerably hinder advancements in related fields. To tackle these challenges, we propose the following: 1) A novel convolution operator named Faster Fourier Convolution (FasterFC), aims at providing an adaptable broad receptive field for spatial domain reconstruction networks with fast computational speed. 2) A split-slice strategy that substantially reduces the computational load of 3D reconstruction, enabling high-resolution, multi-coil, 3D MR image reconstruction while fully utilizing inter-layer and intra-layer information. 3) A single-to-group algorithm that efficiently utilizes scan-specific and data-driven priors to enhance k -space interpolation effects. 4) A multi-stage, multi-coil, 3D fast MRI method, called the faster Fourier convolution based single-to-group network (FAS-Net), comprising a single-to-group k -space interpolation algorithm and a FasterFC-based image domain reconstruction module, significantly minimizes the computational demands of 3D reconstruction through split-slice strategy. Experimental evaluations conducted on the NYU fastMRI and Stanford MRI Data datasets reveal that the FasterFC significantly enhances the quality of both 2D and 3D reconstruction results. Moreover, FAS-Net, characterized as a method that can achieve high-resolution (320, 320, 256), multi-coil, (8 coils), 3D fast MRI, exhibits superior reconstruction performance compared to other state-of-the-art 2D and 3D methods.

Energy-guided diffusion model for CBCT-to-CT synthesis

Cone Beam Computed Tomography (CBCT) plays a crucial role in Image-Guided Radiation Therapy (IGRT), providing essential assurance of accuracy in radiation treatment by monitoring changes in anatomical structures during the treatment process. However, CBCT images often face interference from scatter noise and artifacts, posing a significant challenge when relying solely on CBCT for precise dose calculation and accurate tissue localization. There is an urgent need to enhance the quality of CBCT images, enabling a more practical application in IGRT. This study introduces EGDiff, a novel framework based on the diffusion model, designed to address the challenges posed by scatter noise and artifacts in CBCT images. In our approach, we employ a forward diffusion process by adding Gaussian noise to CT images, followed by a reverse denoising process using ResUNet with an attention mechanism to predict noise intensity, ultimately synthesizing CBCT-to-CT images. Additionally, we design an energy-guided function to retain domain-independent features and discard domain-specific features during the denoising process, enhancing the effectiveness of CBCT-CT generation. We conduct numerous experiments on the thorax dataset and pancreas dataset. The results demonstrate that EGDiff performs better on the thoracic tumor dataset with SSIM of 0.850, MAE of 26.87 HU, PSNR of 19.83 dB, and NCC of 0.874. EGDiff outperforms SoTA CBCT-to-CT synthesis methods on the pancreas dataset with SSIM of 0.754, MAE of 32.19 HU, PSNR of 19.35 dB, and NCC of 0.846. By improving the accuracy and reliability of CBCT images, EGDiff can enhance the precision of radiation therapy, minimize radiation exposure to healthy tissues, and ultimately contribute to more effective and personalized cancer treatment strategies.

Balanced transformer: efficient classification of glioblastoma and primary central nervous system lymphoma

Objective.Primary central nervous system lymphoma (PCNSL) and glioblastoma (GBM) are malignant primary brain tumors with different biological characteristics. Great differences exist between the treatment strategies of PCNSL and GBM. Thus, accurately distinguishing between PCNSL and GBM before surgery is very important for guiding neurosurgery. At present, the spinal fluid of patients is commonly extracted to find tumor markers for diagnosis. However, this method not only causes secondary injury to patients, but also easily delays treatment. Although diagnosis using radiology images is non-invasive, the morphological features and texture features of the two in magnetic resonance imaging (MRI) are quite similar, making distinction with human eyes and image diagnosis very difficult. In order to solve the problem of insufficient number of samples and sample imbalance, we used data augmentation and balanced sample sampling methods. Conventional Transformer networks use patch segmentation operations to divide images into small patches, but the lack of communication between patches leads to unbalanced data layers.Approach.To address this problem, we propose a balanced patch embedding approach that extracts high-level semantic information by reducing the feature dimensionality and maintaining the geometric variation invariance of the features. This approach balances the interactions between the information and improves the representativeness of the data. To further address the imbalance problem, the balanced patch partition method is proposed to increase the receptive field by sampling the four corners of the sliding window and introducing a linear encoding component without increasing the computational effort, and designed a new balanced loss function.Main results.Benefiting from the overall balance design, we conducted an experiment using Balanced Transformer and obtained an accuracy of 99.89%, sensitivity of 99.74%, specificity of 99.73% and AUC of 99.19%, which is far higher than the previous results (accuracy of 89.6% ∼ 96.8%, sensitivity of 74.3% ∼ 91.3%, specificity of 88.9% ∼ 96.02% and AUC of 87.8% ∼ 94.9%).Significance.This study can accurately distinguish PCNSL and GBM before surgery. Because GBM is a common type of malignant tumor, the 1% improvement in accuracy has saved many patients and reduced treatment times considerably. Thus, it can provide doctors with a good basis for auxiliary diagnosis.

Diffusion models in bioinformatics and computational biology

Denoising diffusion models embody a type of generative artificial intelligence that can be applied in computer vision, natural language processing and bioinformatics. In this Review, we introduce the key concepts and theoretical foundations of three diffusion modelling frameworks (denoising diffusion probabilistic models, noise-conditioned scoring networks and score stochastic differential equations). We then explore their applications in bioinformatics and computational biology, including protein design and generation, drug and small-molecule design, protein-ligand interaction modelling, cryo-electron microscopy image data analysis and single-cell data analysis. Finally, we highlight open-source diffusion model tools and consider the future applications of diffusion models in bioinformatics.

Artificial intelligence generated content (AIGC) in medicine: A narrative review

Recently, artificial intelligence generated content (AIGC) has been receiving increased attention and is growing exponentially. AIGC is generated based on the intentional information extracted from human-provided instructions by generative artificial intelligence (AI) models. AIGC quickly and automatically generates large amounts of high-quality content. Currently, there is a shortage of medical resources and complex medical procedures in medicine. Due to its characteristics, AIGC can help alleviate these problems. As a result, the application of AIGC in medicine has gained increased attention in recent years. Therefore, this paper provides a comprehensive review on the recent state of studies involving AIGC in medicine. First, we present an overview of AIGC. Furthermore, based on recent studies, the application of AIGC in medicine is reviewed from two aspects: medical image processing and medical text generation. The basic generative AI models, tasks, target organs, datasets and contribution of studies are considered and summarized. Finally, we also discuss the limitations and challenges faced by AIGC and propose possible solutions with relevant studies. We hope this review can help readers understand the potential of AIGC in medicine and obtain some innovative ideas in this field.

Automatic segmentation of brain glioma based on XY-Net

Glioma is a malignant primary brain tumor, which can easily lead to death if it is not detected in time. Magnetic resonance imaging is the most commonly used technique to diagnose gliomas, and precise outlining of tumor areas from magnetic resonance images (MRIs) is an important aid to physicians in understanding the patient's condition and formulating treatment plans. However, relying on radiologists to manually depict tumors is a tedious and laborious task, so it is clinically important to investigate an automated method for outlining glioma regions in MRIs. To liberate radiologists from the heavy task of outlining tumors, we propose a fully convolutional network, XY-Net, based on the most popular U-Net symmetric encoder-decoder structure to perform automatic segmentation of gliomas. We construct two symmetric sub-encoders for XY-Net and build interconnected X-shaped feature map transmission paths between the sub-encoders, while maintaining the feature map concatenation between each sub-encoder and the decoder. Moreover, a loss function composed of the balanced cross-entropy loss function and the dice loss function is used in the training task of XY-Net to solve the class unevenness problem of the medical image segmentation task. The experimental results show that the proposed XY-Net has a 2.16% improvement in dice coefficient (DC) compared to the network model with a single encoder structure, and compare with some state-of-the-art image segmentation methods, XY-Net achieves the best performance. The DC, HD, recall, and precision of our method on the test set are 74.49%, 10.89 mm, 78.06%, and 76.30%, respectively. The combination of sub-encoders and cross-transmission paths enables the model to perform better; based on this combination, the XY-Net achieves an end-to-end automatic segmentation of gliomas on 2D slices of MRIs, which can play a certain auxiliary role for doctors in grasping the state of illness.

DEQ-MPI: A Deep Equilibrium Reconstruction With Learned Consistency for Magnetic Particle Imaging

Magnetic particle imaging (MPI) offers unparalleled contrast and resolution for tracing magnetic nanoparticles. A common imaging procedure calibrates a system matrix (SM) that is used to reconstruct data from subsequent scans. The ill-posed reconstruction problem can be solved by simultaneously enforcing data consistency based on the SM and regularizing the solution based on an image prior. Traditional hand-crafted priors cannot capture the complex attributes of MPI images, whereas recent MPI methods based on learned priors can suffer from extensive inference times or limited generalization performance. Here, we introduce a novel physics-driven method for MPI reconstruction based on a deep equilibrium model with learned data consistency (DEQ-MPI). DEQ-MPI reconstructs images by augmenting neural networks into an iterative optimization, as inspired by unrolling methods in deep learning. Yet, conventional unrolling methods are computationally restricted to few iterations resulting in non-convergent solutions, and they use hand-crafted consistency measures that can yield suboptimal capture of the data distribution. DEQ-MPI instead trains an implicit mapping to maximize the quality of a convergent solution, and it incorporates a learned consistency measure to better account for the data distribution. Demonstrations on simulated and experimental data indicate that DEQ-MPI achieves superior image quality and competitive inference time to state-of-the-art MPI reconstruction methods.

Parallel-stream fusion of scan-specific and scan-general priors for learning deep MRI reconstruction in low-data regimes

Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan times. Reconstruction methods can alleviate this limitation by recovering clinically usable images from accelerated acquisitions. In particular, learning-based methods promise performance leaps by employing deep neural networks as data-driven priors. A powerful approach uses scan-specific (SS) priors that leverage information regarding the underlying physical signal model for reconstruction. SS priors are learned on each individual test scan without the need for a training dataset, albeit they suffer from computationally burdening inference with nonlinear networks. An alternative approach uses scan-general (SG) priors that instead leverage information regarding the latent features of MRI images for reconstruction. SG priors are frozen at test time for efficiency, albeit they require learning from a large training dataset. Here, we introduce a novel parallel-stream fusion model (PSFNet) that synergistically fuses SS and SG priors for performant MRI reconstruction in low-data regimes, while maintaining competitive inference times to SG methods. PSFNet implements its SG prior based on a nonlinear network, yet it forms its SS prior based on a linear network to maintain efficiency. A pervasive framework for combining multiple priors in MRI reconstruction is algorithmic unrolling that uses serially alternated projections, causing error propagation under low-data regimes. To alleviate error propagation, PSFNet combines its SS and SG priors via a novel parallel-stream architecture with learnable fusion parameters. Demonstrations are performed on multi-coil brain MRI for varying amounts of training data. PSFNet outperforms SG methods in low-data regimes, and surpasses SS methods with few tens of training samples. On average across tasks, PSFNet achieves 3.1 dB higher PSNR, 2.8% higher SSIM, and 0.3 × lower RMSE than baselines. Furthermore, in both supervised and unsupervised setups, PSFNet requires an order of magnitude lower samples compared to SG methods, and enables an order of magnitude faster inference compared to SS methods. Thus, the proposed model improves deep MRI reconstruction with elevated learning and computational efficiency.

Unsupervised Medical Image Translation With Adversarial Diffusion Models

Imputation of missing images via source-to-target modality translation can improve diversity in medical imaging protocols. A pervasive approach for synthesizing target images involves one-shot mapping through generative adversarial networks (GAN). Yet, GAN models that implicitly characterize the image distribution can suffer from limited sample fidelity. Here, we propose a novel method based on adversarial diffusion modeling, SynDiff, for improved performance in medical image translation. To capture a direct correlate of the image distribution, SynDiff leverages a conditional diffusion process that progressively maps noise and source images onto the target image. For fast and accurate image sampling during inference, large diffusion steps are taken with adversarial projections in the reverse diffusion direction. To enable training on unpaired datasets, a cycle-consistent architecture is devised with coupled diffusive and non-diffusive modules that bilaterally translate between two modalities. Extensive assessments are reported on the utility of SynDiff against competing GAN and diffusion models in multi-contrast MRI and MRI-CT translation. Our demonstrations indicate that SynDiff offers quantitatively and qualitatively superior performance against competing baselines.

The past, current, and future of neonatal intensive care units with artificial intelligence: a systematic review

Machine learning and deep learning are two subsets of artificial intelligence that involve teaching computers to learn and make decisions from any sort of data. Most recent developments in artificial intelligence are coming from deep learning, which has proven revolutionary in almost all fields, from computer vision to health sciences. The effects of deep learning in medicine have changed the conventional ways of clinical application significantly. Although some sub-fields of medicine, such as pediatrics, have been relatively slow in receiving the critical benefits of deep learning, related research in pediatrics has started to accumulate to a significant level, too. Hence, in this paper, we review recently developed machine learning and deep learning-based solutions for neonatology applications. We systematically evaluate the roles of both classical machine learning and deep learning in neonatology applications, define the methodologies, including algorithmic developments, and describe the remaining challenges in the assessment of neonatal diseases by using PRISMA 2020 guidelines. To date, the primary areas of focus in neonatology regarding AI applications have included survival analysis, neuroimaging, analysis of vital parameters and biosignals, and retinopathy of prematurity diagnosis. We have categorically summarized 106 research articles from 1996 to 2022 and discussed their pros and cons, respectively. In this systematic review, we aimed to further enhance the comprehensiveness of the study. We also discuss possible directions for new AI models and the future of neonatology with the rising power of AI, suggesting roadmaps for the integration of AI into neonatal intensive care units.

Power-law spectrum-based objective function to train a generative adversarial network with transfer learning for the synthetic breast CT image

Objective.This paper proposes a new objective function to improve the quality of synthesized breast CT images generated by the GAN and compares the GAN performances on transfer learning datasets from different image domains.Approach.The proposed objective function, named beta loss function, is based on the fact that x-ray-based breast images follow the power-law spectrum. Accordingly, the exponent of the power-law spectrum (beta value) for breast CT images is approximately two. The beta loss function is defined in terms of L1 distance between the beta value of synthetic images and validation samples. To compare the GAN performances for transfer learning datasets from different image domains, ImageNet and anatomical noise images are used in the transfer learning dataset. We employ styleGAN2 as the backbone network and add the proposed beta loss function. The patient-derived breast CT dataset is used as the training and validation dataset; 7355 and 212 images are used for network training and validation, respectively. We use the beta value evaluation and Fréchet inception distance (FID) score for quantitative evaluation.Main results.For qualitative assessment, we attempt to replicate the images from the validation dataset using the trained GAN. Our results show that the proposed beta loss function achieves a more similar beta value to real images and a lower FID score. Moreover, we observe that the GAN pretrained with anatomical noise images achieves better equality than ImageNet for beta value evaluation and FID score. Finally, the beta loss function with anatomical noise as the transfer learning dataset achieves the lowest FID score.Significance.Overall, the GAN using the proposed beta loss function with anatomical noise images as the transfer learning dataset provides the lowest FID score among all tested cases. Hence, this work has implications for developing GAN-based breast image synthesis methods for medical imaging applications.

Emerging Trends in Fast MRI Using Deep-Learning Reconstruction on Undersampled k-Space Data: A Systematic Review

Magnetic Resonance Imaging (MRI) is an essential medical imaging modality that provides excellent soft-tissue contrast and high-resolution images of the human body, allowing us to understand detailed information on morphology, structural integrity, and physiologic processes. However, MRI exams usually require lengthy acquisition times. Methods such as parallel MRI and Compressive Sensing (CS) have significantly reduced the MRI acquisition time by acquiring less data through undersampling k-space. The state-of-the-art of fast MRI has recently been redefined by integrating Deep Learning (DL) models with these undersampled approaches. This Systematic Literature Review (SLR) comprehensively analyzes deep MRI reconstruction models, emphasizing the key elements of recently proposed methods and highlighting their strengths and weaknesses. This SLR involves searching and selecting relevant studies from various databases, including Web of Science and Scopus, followed by a rigorous screening and data extraction process using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. It focuses on various techniques, such as residual learning, image representation using encoders and decoders, data-consistency layers, unrolled networks, learned activations, attention modules, plug-and-play priors, diffusion models, and Bayesian methods. This SLR also discusses the use of loss functions and training with adversarial networks to enhance deep MRI reconstruction methods. Moreover, we explore various MRI reconstruction applications, including non-Cartesian reconstruction, super-resolution, dynamic MRI, joint learning of reconstruction with coil sensitivity and sampling, quantitative mapping, and MR fingerprinting. This paper also addresses research questions, provides insights for future directions, and emphasizes robust generalization and artifact handling. Therefore, this SLR serves as a valuable resource for advancing fast MRI, guiding research and development efforts of MRI reconstruction for better image quality and faster data acquisition.