ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Learning Disentangled Priors for Hyperspectral Anomaly Detection: A Coupling Model-Driven and Data-Driven Paradigm

Accurately distinguishing between background and anomalous objects within hyperspectral images poses a significant challenge. The primary obstacle lies in the inadequate modeling of prior knowledge, leading to a performance bottleneck in hyperspectral anomaly detection (HAD). In response to this challenge, we put forth a groundbreaking coupling paradigm that combines model-driven low-rank representation (LRR) methods with data-driven deep learning techniques by learning disentangled priors (LDP). LDP seeks to capture complete priors for effectively modeling the background, thereby extracting anomalies from hyperspectral images more accurately. LDP follows a model-driven deep unfolding architecture, where the prior knowledge is separated into the explicit low-rank prior formulated by expert knowledge and implicit learnable priors by means of deep networks. The internal relationships between explicit and implicit priors within LDP are elegantly modeled through a skip residual connection. Furthermore, we provide a mathematical proof of the convergence of our proposed model. Our experiments, conducted on multiple widely recognized datasets, demonstrate that LDP surpasses most of the current advanced HAD techniques, exceling in both detection performance and generalization capability.

Advancing Hyperspectral and Multispectral Image Fusion: An Information-Aware Transformer-Based Unfolding Network

In hyperspectral image (HSI) processing, the fusion of the high-resolution multispectral image (HR-MSI) and the low-resolution HSI (LR-HSI) on the same scene, known as MSI-HSI fusion, is a crucial step in obtaining the desired high-resolution HSI (HR-HSI). With the powerful representation ability, convolutional neural network (CNN)-based deep unfolding methods have demonstrated promising performances. However, limited receptive fields of CNN often lead to inaccurate long-range spatial features, and inherent input and output images for each stage in unfolding networks restrict the feature transmission, thus limiting the overall performance. To this end, we propose a novel and efficient information-aware transformer-based unfolding network (ITU-Net) to model the long-range dependencies and transfer more information across the stages. Specifically, we employ a customized transformer block to learn representations from both the spatial and frequency domains as well as avoid the quadratic complexity with respect to the input length. For spatial feature extractions, we develop an information transfer guided linearized attention (ITLA), which transmits high-throughput information between adjacent stages and extracts contextual features along the spatial dimension in linear complexity. Moreover, we introduce frequency domain learning in the feedforward network (FFN) to capture token variations of the image and narrow the frequency gap. Via integrating our proposed transformer blocks with the unfolding framework, our ITU-Net achieves state-of-the-art (SOTA) performance on both synthetic and real hyperspectral datasets.

Dual-Ascent-Inspired Transformer for Compressed Sensing

Deep learning has revolutionized image compressed sensing (CS) by enabling lightweight models that achieve high-quality reconstruction with low latency. However, most deep neural network-based CS models are pre-trained for specific compression ratios (CS ratios), limiting their flexibility compared to traditional iterative algorithms. To address this limitation, we propose the Dual-Ascent-Inspired Transformer (DAT), a novel architecture that maintains stable performance across different compression ratios with minimal training costs. DAT's design incorporates the mathematical properties of the dual ascent method (DAM), leading to accelerated training convergence. The architecture features an innovative asymmetric primal-dual space at each iteration layer, enabling dimension-specific operations that balance reconstruction quality with computational efficiency. We also optimize the Cross Attention module through parameter sharing, effectively reducing its training complexity. Experimental results demonstrate DAT's superior performance in two key aspects: First, during early-stage training (within 10 epochs), DAT consistently outperforms existing methods across multiple CS ratios (10%, 30%, and 50%). Notably, DAT achieves comparable PSNR to the ISTA-Net+ baseline within just one epoch, while competing methods require significantly more training time. Second, DAT exhibits enhanced robustness to variations in initial learning rates, as evidenced by loss function analysis during training.

RTF: Recursive TransFusion for Multi-Modal Image Synthesis

Multi-modal image synthesis is crucial for obtaining complete modalities due to the imaging restrictions in reality. Current methods, primarily CNN-based models, find it challenging to extract global representations because of local inductive bias, leading to synthetic structure deformation or color distortion. Despite the significant global representation ability of transformer in capturing long-range dependencies, its huge parameter size requires considerable training data. Multi-modal synthesis solely based on one of the two structures makes it hard to extract comprehensive information from each modality with limited data. To tackle this dilemma, we propose a simple yet effective Recursive TransFusion (RTF) framework for multi-modal image synthesis. Specifically, we develop a TransFusion unit to integrate local knowledge extracted from the individual modality by connecting a CNN-based local representation block (LRB) and a transformer-based global fusion block (GFB) via a feature translating gate (FTG). Considering the numerous parameters introduced by the transformer, we further unfold a TransFusion unit with recursive constraint repeatedly, forming recursive TransFusion (RTF), which progressively extracts multi-modal information at different depths. Our RTF remarkably reduces network parameters while maintaining superior performance. Extensive experiments validate our superiority against the competing methods on multiple benchmarks. The source code will be available at https://github.com/guoliangq/RTF.

CAMP-Net: Consistency-Aware Multi-Prior Network for Accelerated MRI Reconstruction

Undersampling -space data in magnetic resonance imaging (MRI) reduces scan time but pose challenges in image reconstruction. Considerable progress has been made in reconstructing accelerated MRI. However, restoration of high-frequency image details in highly undersampled data remains challenging. To address this issue, we propose CAMP-Net, an unrolling-based Consistency-Aware Multi-Prior Network for accelerated MRI reconstruction. CAMP-Net leverages complementary multi-prior knowledge and multi-slice information from various domains to enhance reconstruction quality. Specifically, CAMP-Net comprises three interleaved modules for image enhancement, -space restoration, and calibration consistency, respectively. These modules jointly learn priors from data in image domain, -domain, and calibration region, respectively, in data-driven manner during each unrolled iteration. Notably, the encoded calibration prior knowledge extracted from auto-calibrating signals implicitly guides the learning of consistency-aware -space correlation for reliable interpolation of missing -space data. To maximize the benefits of image domain and -domain prior knowledge, the reconstructions are aggregated in a frequency fusion module, exploiting their complementary properties to optimize the trade-off between artifact removal and fine detail preservation. Additionally, we incorporate a surface data fidelity layer during the learning of -domain and calibration domain priors to prevent degradation of the reconstruction caused by padding-induced data imperfections. We evaluate the generalizability and robustness of our method on three large public datasets with varying acceleration factors and sampling patterns. The experimental results demonstrate that our method outperforms state-of-the-art approaches in terms of both reconstruction quality and mapping estimation, particularly in scenarios with high acceleration factors.

DONet: Dual-Octave Network for Fast MR Image Reconstruction

Magnetic resonance (MR) image acquisition is an inherently prolonged process, whose acceleration has long been the subject of research. This is commonly achieved by obtaining multiple undersampled images, simultaneously, through parallel imaging. In this article, we propose the dual-octave network (DONet), which is capable of learning multiscale spatial-frequency features from both the real and imaginary components of MR data, for parallel fast MR image reconstruction. More specifically, our DONet consists of a series of dual-octave convolutions (Dual-OctConvs), which are connected in a dense manner for better reuse of features. In each Dual-OctConv, the input feature maps and convolutional kernels are first split into two components (i.e., real and imaginary) and then divided into four groups according to their spatial frequencies. Then, our Dual-OctConv conducts intragroup information updating and intergroup information exchange to aggregate the contextual information across different groups. Our framework provides three appealing benefits: 1) it encourages information interaction and fusion between the real and imaginary components at various spatial frequencies to achieve richer representational capacity; 2) the dense connections between the real and imaginary groups in each Dual-OctConv make the propagation of features more efficient by feature reuse; and 3) DONet enlarges the receptive field by learning multiple spatial-frequency features of both the real and imaginary components. Extensive experiments on two popular datasets (i.e., clinical knee and fastMRI), under different undersampling patterns and acceleration factors, demonstrate the superiority of our model in accelerated parallel MR image reconstruction.

DEs-Inspired Accelerated Unfolded Linearized ADMM Networks for Inverse Problems

Many research works have shown that the traditional alternating direction multiplier methods (ADMMs) can be better understood by continuous-time differential equations (DEs). On the other hand, many unfolded algorithms directly inherit the traditional iterations to build deep networks. Although they achieve superior practical performance and a faster convergence rate than traditional counterparts, there is a lack of clear insight into unfolded network structures. Thus, we attempt to explore the unfolded linearized ADMM (LADMM) from the perspective of DEs, and design more efficient unfolded networks. First, by proposing an unfolded Euler LADMM scheme and inspired by the trapezoid discretization, we design a new more accurate Trapezoid LADMM scheme. For the convenience of implementation, we provide its explicit version via a prediction-correction strategy. Then, to expand the representation space of unfolded networks, we design an accelerated variant of our Euler LADMM scheme, which can be interpreted as second-order DEs with stronger representation capabilities. To fully explore this representation space, we designed an accelerated Trapezoid LADMM scheme. To the best of our knowledge, this is the first work to explore a comprehensive connection with theoretical guarantees between unfolded ADMMs and first- (second-) order DEs. Finally, we instantiate our schemes as (A-)ELADMM and (A-)TLADMM with the proximal operators, and (A-)ELADMM-Net and (A-)TLADMM-Net with convolutional neural networks (CNNs). Extensive inverse problem experiments show that our Trapezoid LADMM schemes perform better than well-known methods.

SSM-Net: Enhancing Compressed Sensing Image Reconstruction with Mamba Architecture and Fast Iterative Shrinking Threshold Algorithm Optimization

Compressed sensing (CS) is a powerful technique that can reduce data size while maintaining high reconstruction quality, which makes it particularly valuable in high-dimensional image applications. However, many existing methods have difficulty balancing reconstruction accuracy, computational efficiency, and fast convergence. To address these challenges, this paper proposes SSM-Net, a novel framework that combines the state-space modeling (SSM) of the Mamba architecture with the fast iterative shrinking threshold algorithm (FISTA). The Mamba-based SSM module can effectively capture local and global dependencies with linear computational complexity and significantly reduces the computation time compared to Transformer-based methods. In addition, the momentum update inspired by FISTA improves the convergence speed during deep iterative reconstruction. SSM-Net features a lightweight sampling module for efficient data compression, an initial reconstruction module for fast approximation, and a deep reconstruction module for iterative refinement. Extensive experiments on various benchmark datasets show that SSM-Net achieves state-of-the-art reconstruction performance while reducing both training and inference reconstruction time, making SSM-Net a scalable and practical solution for real-time applications of compressed sensing.

SPIRiT-Diffusion: Self-Consistency Driven Diffusion Model for Accelerated MRI

Diffusion models have emerged as a leading methodology for image generation and have proven successful in the realm of magnetic resonance imaging (MRI) reconstruction. However, existing reconstruction methods based on diffusion models are primarily formulated in the image domain, making the reconstruction quality susceptible to inaccuracies in coil sensitivity maps (CSMs). k-space interpolation methods can effectively address this issue but conventional diffusion models are not readily applicable in k-space interpolation. To overcome this challenge, we introduce a novel approach called SPIRiT-Diffusion, which is a diffusion model for k-space interpolation inspired by the iterative self-consistent SPIRiT method. Specifically, we utilize the iterative solver of the self-consistent term (i.e., k-space physical prior) in SPIRiT to formulate a novel stochastic differential equation (SDE) governing the diffusion process. Subsequently, k-space data can be interpolated by executing the diffusion process. This innovative approach highlights the optimization model's role in designing the SDE in diffusion models, enabling the diffusion process to align closely with the physics inherent in the optimization model-a concept referred to as model-driven diffusion. We evaluated the proposed SPIRiT-Diffusion method using a 3D joint intracranial and carotid vessel wall imaging dataset. The results convincingly demonstrate its superiority over image-domain reconstruction methods, achieving high reconstruction quality even at a substantial acceleration rate of 10. Our code are available at https://github.com/zhyjSIAT/SPIRiT-Diffusion.

Accelerating multi-coil MR image reconstruction using weak supervision

Deep-learning-based MR image reconstruction in settings where large fully sampled dataset collection is infeasible requires methods that effectively use both under-sampled and fully sampled datasets. This paper evaluates a weakly supervised, multi-coil, physics-guided approach to MR image reconstruction, leveraging both dataset types, to improve both the quality and robustness of reconstruction. A physics-guided end-to-end variational network (VarNet) is pretrained in a self-supervised manner using a 4 × under-sampled dataset following the self-supervised learning via data undersampling (SSDU) methodology. The pre-trained weights are transferred to another VarNet, which is fine-tuned using a smaller, fully sampled dataset by optimizing multi-scale structural similarity (MS-SSIM) loss in image space. The proposed methodology is compared with fully self-supervised and fully supervised training. Reconstruction quality improvements in SSIM, PSNR, and NRMSE when abundant training data is available (the high-data regime), and enhanced robustness when training data is scarce (the low-data regime) are demonstrated using weak supervision for knee and brain MR image reconstructions at 8 × and 10 × acceleration, respectively. Multi-coil physics-guided MR image reconstruction using both under-sampled and fully sampled datasets is achievable with transfer learning and fine-tuning. This methodology can provide improved reconstruction quality in the high-data regime and improved robustness in the low-data regime at high acceleration rates.

USB-Net: Unfolding Split Bregman Method With Multi-Phase Feature Integration for Compressive Imaging

Existing unfolding-based compressive imaging approaches always suffer from certain issues, including inefficient feature extraction and information loss during iterative reconstruction phases, which become particularly evident at low sampling ratios, i.e., significant detail degradation and distortion in reconstructed images. To mitigate these challenges, we propose USB-Net, a deep unfolding method inspired by the renowned Split Bregman algorithm and multi-phase feature integration strategy, for compressive imaging reconstruction. Specifically, we use a customized Depthwise Attention Block as a fundamental block for feature extraction, but also to address the sparse induction-related splitting operator within Split Bregman method. Based on this, we introduce three Auxiliary Iteration Modules: X(k), D(k), and B(k) to reinforce the effectiveness of Split Bregman's decomposition strategy for problem breakdown and Bregman iterations. Moreover, we introduce two categories of Iterative Fusion Modules to seamlessly harmonize and integrate insights across iterative reconstruction phases, enhancing the utilization of crucial features, such as edge information and textures. In general, USB-Net can fully harness the advantages of traditional Split Bregman approach, manipulating multi-phase iterative insights to enhance feature extraction, optimize data fidelity, and achieve high-quality image reconstruction. Extensive experiments show that USB-Net significantly outperforms current state-of-the-art methods on image compressive sensing, CS-magnetic resonance imaging, and snapshot compressive imaging tasks, demonstrating superior generalizability. Our code is available at USB-Net.

Deep unfolding network with spatial alignment for multi-modal MRI reconstruction

Multi-modal Magnetic Resonance Imaging (MRI) offers complementary diagnostic information, but some modalities are limited by the long scanning time. To accelerate the whole acquisition process, MRI reconstruction of one modality from highly under-sampled k-space data with another fully-sampled reference modality is an efficient solution. However, the misalignment between modalities, which is common in clinic practice, can negatively affect reconstruction quality. Existing deep learning-based methods that account for inter-modality misalignment perform better, but still share two main common limitations: (1) The spatial alignment task is not adaptively integrated with the reconstruction process, resulting in insufficient complementarity between the two tasks; (2) the entire framework has weak interpretability. In this paper, we construct a novel Deep Unfolding Network with Spatial Alignment, termed DUN-SA, to appropriately embed the spatial alignment task into the reconstruction process. Concretely, we derive a novel joint alignment-reconstruction model with a specially designed aligned cross-modal prior term. By relaxing the model into cross-modal spatial alignment and multi-modal reconstruction tasks, we propose an effective algorithm to solve this model alternatively. Then, we unfold the iterative stages of the proposed algorithm and design corresponding network modules to build DUN-SA with interpretability. Through end-to-end training, we effectively compensate for spatial misalignment using only reconstruction loss, and utilize the progressively aligned reference modality to provide inter-modality prior to improve the reconstruction of the target modality. Comprehensive experiments on four real datasets demonstrate that our method exhibits superior reconstruction performance compared to state-of-the-art methods.

Enhancing global sensitivity and uncertainty quantification in medical image reconstruction with Monte Carlo arbitrary-masked mamba

Deep learning has been extensively applied in medical image reconstruction, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) represent the predominant paradigms, each possessing distinct advantages and inherent limitations: CNNs exhibit linear complexity with local sensitivity, whereas ViTs demonstrate quadratic complexity with global sensitivity. The emerging Mamba has shown superiority in learning visual representation, which combines the advantages of linear scalability and global sensitivity. In this study, we introduce MambaMIR, an Arbitrary-Masked Mamba-based model with wavelet decomposition for joint medical image reconstruction and uncertainty estimation. A novel Arbitrary Scan Masking (ASM) mechanism "masks out" redundant information to introduce randomness for further uncertainty estimation. Compared to the commonly used Monte Carlo (MC) dropout, our proposed MC-ASM provides an uncertainty map without the need for hyperparameter tuning and mitigates the performance drop typically observed when applying dropout to low-level tasks. For further texture preservation and better perceptual quality, we employ the wavelet transformation into MambaMIR and explore its variant based on the Generative Adversarial Network, namely MambaMIR-GAN. Comprehensive experiments have been conducted for multiple representative medical image reconstruction tasks, demonstrating that the proposed MambaMIR and MambaMIR-GAN outperform other baseline and state-of-the-art methods in different reconstruction tasks, where MambaMIR achieves the best reconstruction fidelity and MambaMIR-GAN has the best perceptual quality. In addition, our MC-ASM provides uncertainty maps as an additional tool for clinicians, while mitigating the typical performance drop caused by the commonly used dropout.

Fast and Reliable Score-Based Generative Model for Parallel MRI

The score-based generative model (SGM) can generate high-quality samples, which have been successfully adopted for magnetic resonance imaging (MRI) reconstruction. However, the recent SGMs may take thousands of steps to generate a high-quality image. Besides, SGMs neglect to exploit the redundancy in space. To overcome the above two drawbacks, in this article, we propose a fast and reliable SGM (FRSGM). First, we propose deep ensemble denoisers (DEDs) consisting of SGM and the deep denoiser, which are used to solve the proximal problem of the implicit regularization term. Second, we propose a spatially adaptive self-consistency (SASC) term as the regularization term of the -space data. We use the alternating direction method of multipliers (ADMM) algorithm to solve the minimization model of compressed sensing (CS)-MRI incorporating the image prior term and the SASC term, which is significantly faster than the related works based on SGM. Meanwhile, we can prove that the iterating sequence of the proposed algorithm has a unique fixed point. In addition, the DED and the SASC term can significantly improve the generalization ability of the algorithm. The features mentioned above make our algorithm reliable, including the fixed-point convergence guarantee, the exploitation of the space, and the powerful generalization ability.

Generalizable Reconstruction for Accelerating MR Imaging via Federated Learning With Neural Architecture Search

Heterogeneous data captured by different scanning devices and imaging protocols can affect the generalization performance of the deep learning magnetic resonance (MR) reconstruction model. While a centralized training model is effective in mitigating this problem, it raises concerns about privacy protection. Federated learning is a distributed training paradigm that can utilize multi-institutional data for collaborative training without sharing data. However, existing federated learning MR image reconstruction methods rely on models designed manually by experts, which are complex and computationally expensive, suffering from performance degradation when facing heterogeneous data distributions. In addition, these methods give inadequate consideration to fairness issues, namely ensuring that the model's training does not introduce bias towards any specific dataset's distribution. To this end, this paper proposes a generalizable federated neural architecture search framework for accelerating MR imaging (GAutoMRI). Specifically, automatic neural architecture search is investigated for effective and efficient neural network representation learning of MR images from different centers. Furthermore, we design a fairness adjustment approach that can enable the model to learn features fairly from inconsistent distributions of different devices and centers, and thus facilitate the model to generalize well to the unseen center. Extensive experiments show that our proposed GAutoMRI has better performances and generalization ability compared with seven state-of-the-art federated learning methods. Moreover, the GAutoMRI model is significantly more lightweight, making it an efficient choice for MR image reconstruction tasks. The code will be made available at https://github.com/ternencewu123/GAutoMRI.

AutoSamp: Autoencoding k-Space Sampling via Variational Information Maximization for 3D MRI

Accelerated MRI protocols routinely involve a predefined sampling pattern that undersamples the k-space. Finding an optimal pattern can enhance the reconstruction quality, however this optimization is a challenging task. To address this challenge, we introduce a novel deep learning framework, AutoSamp, based on variational information maximization that enables joint optimization of sampling pattern and reconstruction of MRI scans. We represent the encoder as a non-uniform Fast Fourier Transform that allows continuous optimization of k-space sample locations on a non-Cartesian plane, and the decoder as a deep reconstruction network. Experiments on public 3D acquired MRI datasets show improved reconstruction quality of the proposed AutoSamp method over the prevailing variable density and variable density Poisson disc sampling for both compressed sensing and deep learning reconstructions. We demonstrate that our data-driven sampling optimization method achieves 4.4dB, 2.0dB, 0.75dB, 0.7dB PSNR improvements over reconstruction with Poisson Disc masks for acceleration factors of R =5, 10, 15, 25, respectively. Prospectively accelerated acquisitions with 3D FSE sequences using our optimized sampling patterns exhibit improved image quality and sharpness. Furthermore, we analyze the characteristics of the learned sampling patterns with respect to changes in acceleration factor, measurement noise, underlying anatomy, and coil sensitivities. We show that all these factors contribute to the optimization result by affecting the sampling density, k-space coverage and point spread functions of the learned sampling patterns.

Adaptive Memory-Augmented Unfolding Network for Compressed Sensing

Deep unfolding networks (DUNs) have attracted growing attention in compressed sensing (CS) due to their good interpretability and high performance. However, many DUNs often improve the reconstruction effect at the price of a large number of parameters and have the problem of feature information loss during iteration. This paper proposes a novel adaptive memory-augmented unfolding network for compressed sensing (AMAUN-CS). Concretely, without loss of interpretability, we integrate an adaptive content-aware strategy into the gradient descent step of the proximal gradient descent (PGD) algorithm, driving it to adaptively capture the adequate features. In addition, we extended AMAUN-CS based on the memory storage mechanism of the human brain and propose AMAUN-CS+ to develop the dependency of deep information across cascading stages. The experimental results show that the AMAUN-CS model surpasses other advanced methods on various public benchmark datasets while having lower complexity in training.

ACGRHA-Net: Accelerated multi-contrast MR imaging with adjacency complementary graph assisted residual hybrid attention network

Multi-contrast magnetic resonance (MR) imaging is an advanced technology used in medical diagnosis, but the long acquisition process can lead to patient discomfort and limit its broader application. Shortening acquisition time by undersampling k-space data introduces noticeable aliasing artifacts. To address this, we propose a method that reconstructs multi-contrast MR images from zero-filled data by utilizing a fully-sampled auxiliary contrast MR image as a prior to learn an adjacency complementary graph. This graph is then combined with a residual hybrid attention network, forming the adjacency complementary graph assisted residual hybrid attention network (ACGRHA-Net) for multi-contrast MR image reconstruction. Specifically, the optimal structural similarity is represented by a graph learned from the fully sampled auxiliary image, where the node features and adjacency matrices are designed to precisely capture structural information among different contrast images. This structural similarity enables effective fusion with the target image, improving the detail reconstruction. Additionally, a residual hybrid attention module is designed in parallel with the graph convolution network, allowing it to effectively capture key features and adaptively emphasize these important features in target contrast MR images. This strategy prioritizes crucial information while preserving shallow features, thereby achieving comprehensive feature fusion at deeper levels to enhance multi-contrast MR image reconstruction. Extensive experiments on the different datasets, using various sampling patterns and accelerated factors demonstrate that the proposed method outperforms the current state-of-the-art reconstruction methods.

A review of deep learning-based reconstruction methods for accelerated MRI using spatiotemporal and multi-contrast redundancies

Accelerated magnetic resonance imaging (MRI) has played an essential role in reducing data acquisition time for MRI. Acceleration can be achieved by acquiring fewer data points in k-space, which results in various artifacts in the image domain. Conventional reconstruction methods have resolved the artifacts by utilizing multi-coil information, but with limited robustness. Recently, numerous deep learning-based reconstruction methods have been developed, enabling outstanding reconstruction performances with higher acceleration. Advances in hardware and developments of specialized network architectures have produced such achievements. Besides, MRI signals contain various redundant information including multi-coil redundancy, multi-contrast redundancy, and spatiotemporal redundancy. Utilization of the redundant information combined with deep learning approaches allow not only higher acceleration, but also well-preserved details in the reconstructed images. Consequently, this review paper introduces the basic concepts of deep learning and conventional accelerated MRI reconstruction methods, followed by review of recent deep learning-based reconstruction methods that exploit various redundancies. Lastly, the paper concludes by discussing the challenges, limitations, and potential directions of future developments.

LD-CSNet: A latent diffusion-based architecture for perceptual Compressed Sensing

Compressed Sensing (CS) is a groundbreaking paradigm in image acquisition, challenging the constraints of the Nyquist-Shannon sampling theorem. This enables high-quality image reconstruction using a minimal number of measurements. Neural Networks' potent feature induction capabilities enable advanced data-driven CS methods to achieve high-fidelity image reconstruction. However, achieving satisfactory reconstruction performance, particularly in terms of perceptual quality, remains challenging at extremely low sampling rates. To tackle this challenge, we introduce a novel two-stage image CS framework based on latent diffusion, named LD-CSNet. In the first stage, we utilize an autoencoder pre-trained on a large dataset to represent natural images as low-dimensional latent vectors, establishing prior knowledge distinct from sparsity and effectively reducing the dimensionality of the solution space. In the second stage, we employ a conditional diffusion model for maximum likelihood estimates in the latent space. This is supported by a measurement embedding module designed to encode measurements, making them suitable for a denoising network. This guides the generation process in reconstructing low-dimensional latent vectors. Finally, the image is reconstructed using a pre-trained decoder. Experimental results across multiple public datasets demonstrate LD-CSNet's superior perceptual quality and robustness to noise. It maintains fidelity and visual quality at lower sampling rates. Research findings suggest the promising application of diffusion models in image CS. Future research can focus on developing more appropriate models for the first stage.

Cross-Modal Vertical Federated Learning for MRI Reconstruction

Federated learning enables multiple hospitals to cooperatively learn a shared model without privacy disclosure. Existing methods often take a common assumption that the data from different hospitals have the same modalities. However, such a setting is difficult to fully satisfy in practical applications, since the imaging guidelines may be different between hospitals, which makes the number of individuals with the same set of modalities limited. To this end, we formulate this practical-yet-challenging cross-modal vertical federated learning task, in which data from multiple hospitals have different modalities with a small amount of multi-modality data collected from the same individuals. To tackle such a situation, we develop a novel framework, namely Federated Consistent Regularization constrained Feature Disentanglement (Fed-CRFD), for boosting MRI reconstruction by effectively exploring the overlapping samples (i.e., same patients with different modalities at different hospitals) and solving the domain shift problem caused by different modalities. Particularly, our Fed-CRFD involves an intra-client feature disentangle scheme to decouple data into modality-invariant and modality-specific features, where the modality-invariant features are leveraged to mitigate the domain shift problem. In addition, a cross-client latent representation consistency constraint is proposed specifically for the overlapping samples to further align the modality-invariant features extracted from different modalities. Hence, our method can fully exploit the multi-source data from hospitals while alleviating the domain shift problem. Extensive experiments on two typical MRI datasets demonstrate that our network clearly outperforms state-of-the-art MRI reconstruction methods.

Complex-valued image reconstruction for compressed sensing MRI using hierarchical constraint

In Magnetic Resonance Imaging (MRI), the sequential acquisition of raw complex-valued image data in Fourier space, also known as k-space, results in extended examination times. To speed up the MRI scans, k-space data are usually undersampled and processed using numerical techniques such as compressed sensing (CS). While the majority of CS-MRI algorithms primarily focus on magnitude images due to their significant diagnostic value, the phase components of complex-valued MRI images also hold substantial importance for clinical diagnosis, including neurodegenerative diseases. In this work, complex-valued MRI reconstruction is studied with a focus on the simultaneous reconstruction of both magnitude and phase images. The proposed algorithm is based on the nonsubsampled contourlet transform (NSCT) technique, which offers shift invariance in images. Instead of directly transforming the complex-valued image into the NSCT domain, we introduce a wavelet transform within the NSCT domain, reducing the size of the sparsity of coefficients. This two-level hierarchical constraint (HC) enforces sparse representation of complex-valued images for CS-MRI implementation. The proposed HC is seamlessly integrated into a proximal algorithm simultaneously. Additionally, to effectively minimize the artifacts caused by sub-sampling, thresholds related to different sub-bands in the HC are applied through an alternating optimization process. Experimental results show that the novel method outperforms existing CS-MRI techniques in phase-regularized complex-valued image reconstructions.

Wavelet-Inspired Multi-Channel Score-Based Model for Limited-Angle CT Reconstruction

Score-based generative model (SGM) has demonstrated great potential in the challenging limited-angle CT (LA-CT) reconstruction. SGM essentially models the probability density of the ground truth data and generates reconstruction results by sampling from it. Nevertheless, direct application of the existing SGM methods to LA-CT suffers multiple limitations. Firstly, the directional distribution of the artifacts attributing to the missing angles is ignored. Secondly, the different distribution properties of the artifacts in different frequency components have not been fully explored. These drawbacks would inevitably degrade the estimation of the probability density and the reconstruction results. After an in-depth analysis of these factors, this paper proposes a Wavelet-Inspired Score-based Model (WISM) for LA-CT reconstruction. Specifically, besides training a typical SGM with the original images, the proposed method additionally performs the wavelet transform and models the probability density in each wavelet component with an extra SGM. The wavelet components preserve the spatial correspondence with the original image while performing frequency decomposition, thereby keeping the directional property of the artifacts for further analysis. On the other hand, different wavelet components possess more specific contents of the original image in different frequency ranges, simplifying the probability density modeling by decomposing the overall density into component-wise ones. The resulting two SGMs in the image-domain and wavelet-domain are integrated into a unified sampling process under the guidance of the observation data, jointly generating high-quality and consistent LA-CT reconstructions. The experimental evaluation on various datasets consistently verifies the superior performance of the proposed method over the competing method.

FusionOpt-Net: A Transformer-Based Compressive Sensing Reconstruction Algorithm

Compressive sensing (CS) is a notable technique in signal processing, especially in multimedia, as it allows for simultaneous signal acquisition and dimensionality reduction. Recent advancements in deep learning (DL) have led to the creation of deep unfolding architectures, which overcome the inefficiency and subpar quality of traditional CS reconstruction methods. In this paper, we introduce a novel CS image reconstruction algorithm that leverages the strengths of the fast iterative shrinkage-thresholding algorithm (FISTA) and modern Transformer networks. To enhance computational efficiency, we employ a block-based sampling approach in the sampling module. By mapping FISTA's iterative process onto neural networks in the reconstruction module, we address the hyperparameter challenges of traditional algorithms, thereby improving reconstruction efficiency. Moreover, the robust feature extraction capabilities of Transformer networks significantly enhance image reconstruction quality. Experimental results show that the FusionOpt-Net model surpasses other advanced methods on various public benchmark datasets.

Designing Universally-Approximating Deep Neural Networks: A First-Order Optimization Approach

Universal approximation capability, also referred to as universality, is an important property of deep neural networks, endowing them with the potency to accurately represent the underlying target function in learning tasks. In practice, the architecture of deep neural networks largely influences the performance of the models. However, most existing methodologies for designing neural architectures, such as the heuristic manual design or neural architecture search, ignore the universal approximation property, thus losing a potential safeguard about the performance. In this paper, we propose a unified framework to design the architectures of deep neural networks with a universality guarantee based on first-order optimization algorithms, where the forward pass is interpreted as the updates of an optimization algorithm. The (explicit or implicit) network is designed by replacing each gradient term in the algorithm with a learnable module similar to a two-layer network or its derivatives. Specifically, we explore the realm of width-bounded neural networks, a common practical scenario, showcasing their universality. Moreover, adding operations of normalization, downsampling, and upsampling does not hurt the universality. To the best of our knowledge, this is the first work that width-bounded networks with universal approximation guarantee can be designed in a principled way. Our framework can inspire a variety of neural architectures including some renowned structures such as ResNet and DenseNet, as well as novel innovations. The experimental results on image classification problems demonstrate that the newly inspired networks are competitive and surpass the baselines of ResNet, DenseNet, as well as the advanced ConvNeXt and ViT, testifying to the effectiveness of our framework.

Knowledge-driven deep learning for fast MR imaging: Undersampled MR image reconstruction from supervised to un-supervised learning

Deep learning (DL) has emerged as a leading approach in accelerating MRI. It employs deep neural networks to extract knowledge from available datasets and then applies the trained networks to reconstruct accurate images from limited measurements. Unlike natural image restoration problems, MRI involves physics-based imaging processes, unique data properties, and diverse imaging tasks. This domain knowledge needs to be integrated with data-driven approaches. Our review will introduce the significant challenges faced by such knowledge-driven DL approaches in the context of fast MRI along with several notable solutions, which include learning neural networks and addressing different imaging application scenarios. The traits and trends of these techniques have also been given which have shifted from supervised learning to semi-supervised learning, and finally, to unsupervised learning methods. In addition, MR vendors' choices of DL reconstruction have been provided along with some discussions on open questions and future directions, which are critical for the reliable imaging systems.

Image reconstruction of multispectral sparse sampling photoacoustic tomography based on deep algorithm unrolling

Photoacoustic tomography (PAT), as a novel medical imaging technology, provides structural, functional, and metabolism information of biological tissue in vivo. Sparse Sampling PAT, or SS-PAT, generates images with a smaller number of detectors, yet its image reconstruction is inherently ill-posed. Model-based methods are the state-of-the-art method for SS-PAT image reconstruction, but they require design of complex handcrafted prior. Owing to their ability to derive robust prior from labeled datasets, deep-learning-based methods have achieved great success in solving inverse problems, yet their interpretability is poor. Herein, we propose a novel SS-PAT image reconstruction method based on deep algorithm unrolling (DAU), which integrates the advantages of model-based and deep-learning-based methods. We firstly provide a thorough analysis of DAU for PAT reconstruction. Then, in order to incorporate the structural prior constraint, we propose a nested DAU framework based on plug-and-play Alternating Direction Method of Multipliers (PnP-ADMM) to deal with the sparse sampling problem. Experimental results on numerical simulation, in vivo animal imaging, and multispectral un-mixing demonstrate that the proposed DAU image reconstruction framework outperforms state-of-the-art model-based and deep-learning-based methods.

Compressed-sensing-inspired reconstruction algorithms in low-dose computed tomography: A review

BACKGROUND

Optimization of the dose the patient receives during scanning is an important problem in modern medical X-ray computed tomography (CT). One of the basic ways to its solution is to reduce the number of views. Compressed sensing theory helped promote the development of a new class of effective reconstruction algorithms for limited data CT. These compressed-sensing-inspired (CSI) algorithms optimize the Lp (0 ≤ p ≤ 1) norm of images and can accurately reconstruct CT tomograms from a very few views. The paper presents a review of the CSI algorithms and discusses prospects for their further use in commercial low-dose CT.

METHODS

Many literature references with the CSI algorithms have been were searched. To structure the material collected the author gives a classification framework within which he describes Lp regularization methods, the basic CSI algorithms that are used most often in few-view CT, and some of their derivatives. Lots of examples are provided to illustrate the use of the CSI algorithms in few-view and low-dose CT.

RESULTS

A list of the CSI algorithms is compiled from the literature search. For better demonstrativeness they are summarized in a table. The inference is done that already today some of the algorithms are capable of reconstruction from 20 to 30 views with acceptable quality and dose reduction by a factor of 10.

DISCUSSION

In conclusion the author discusses how soon the CSI reconstruction algorithms can be introduced in the practice of medical diagnosis and used in commercial CT scanners.

Imaging quality enhancement in photon-counting single-pixel imaging via an ADMM-based deep unfolding network in small animal fluorescence imaging

Photon-counting single-pixel imaging (SPI) can image under low-light conditions with high-sensitivity detection. However, the imaging quality of these systems will degrade due to the undersampling and intrinsic photon-noise in practical applications. Here, we propose a deep unfolding network based on the Bayesian maximum a posterior (MAP) estimation and alternating direction method of multipliers (ADMM) algorithm. The reconstruction framework adopts a learnable denoiser by convolutional neural network (CNN) instead of explicit function with hand-crafted prior. Our method enhances the imaging quality compared to traditional methods and data-driven CNN under different photon-noise levels at a low sampling rate of 8%. Using our method, the sensitivity of photon-counting SPI prototype system for fluorescence imaging can reach 7.4 pmol/ml. In-vivo imaging of a mouse bearing tumor demonstrates an 8-times imaging efficiency improvement.

Adaptive High-frequency Enhancement Network with Equilibrated Mechanism for MR Imaging

Unrolling is a promising deep learning (DL)-based technique in accelerating Magnetic Resonance (MR) imaging. It incorporates deep learning networks into the iterative optimization algorithm, enabling the flexible and adaptive learning of undefined parameters or functions. However, unrolling methods employ a limited number of iterations owing to memory constraints and exhibit detail loss, which hinders their overall performance and practical applications. In this work, we propose a novel network that adaptively improves the high-frequency feature representation and integrates a deep equilibrium model for fixed-point iteration to enhance the robustness of the reconstruction. Experimental results demonstrate superior performance of the proposed method in detail preservation, generalization across various test datasets, and robustness against noise interference and number of iterations.

Integrating data distribution prior via Langevin dynamics for end-to-end MR reconstruction

PURPOSE

To develop a novel deep learning-based method inheriting the advantages of data distribution prior and end-to-end training for accelerating MRI.

METHODS

Langevin dynamics is used to formulate image reconstruction with data distribution before facilitate image reconstruction. The data distribution prior is learned implicitly through the end-to-end adversarial training to mitigate the hyper-parameter selection and shorten the testing time compared to traditional probabilistic reconstruction. By seamlessly integrating the deep equilibrium model, the iteration of Langevin dynamics culminates in convergence to a fix-point, ensuring the stability of the learned distribution.

RESULTS

The feasibility of the proposed method is evaluated on the brain and knee datasets. Retrospective results with uniform and random masks show that the proposed method demonstrates superior performance both quantitatively and qualitatively than the state-of-the-art.

CONCLUSION

The proposed method incorporating Langevin dynamics with end-to-end adversarial training facilitates efficient and robust reconstruction for MRI. Empirical evaluations conducted on brain and knee datasets compellingly demonstrate the superior performance of the proposed method in terms of artifact removing and detail preserving.

Single-pixel Fresnel incoherent correlation holography compressed imaging using a Trumpet network

Fresnel incoherent correlation holography (FINCH) can achieve high-precision and non-scanning 3D imaging. However, as a holographic imaging technology, the huge bandwidth requirements and the amount of holographic data transmitted have always been one of the important factors limiting its application. In addition, the hardware cost of pixel array-based CCD or CMOS imaging is very high under high resolution or specific wavelength conditions. Accordingly, a single-pixel Fresnel incoherent correlation holography (SP-FINCH) compressed imaging method is proposed, which replaces pixel array detector with single-pixel detector and designs a Trumpet network to achieve low-cost and high-resolution imaging. Firstly, a modified FINCH imaging system is constructed and data acquisition is carried out using a single-pixel detector. Secondly, a Trumpet network is constructed to directly map the relationship between one-dimensional sampled data and two-dimensional image in an end-to-end manner. Moreover, by comparing the reconstructed images using neural network with that using commonly used single-pixel reconstruction methods, the results indicate that the proposed SP-FINCH compressed imaging method can significantly improve the quality of image reconstruction at lower sampling rate and achieve imaging without phase-shifting operation. The proposed method has been shown to be feasible and advantageous through numerical simulations and optical experiment results.

Fourier Domain Robust Denoising Decomposition and Adaptive Patch MRI Reconstruction

The sparsity of the Fourier transform domain has been applied to magnetic resonance imaging (MRI) reconstruction in k -space. Although unsupervised adaptive patch optimization methods have shown promise compared to data-driven-based supervised methods, the following challenges exist in MRI reconstruction: 1) in previous k -space MRI reconstruction tasks, MRI with noise interference in the acquisition process is rarely considered. 2) Differences in transform domains should be resolved to achieve the high-quality reconstruction of low undersampled MRI data. 3) Robust patch dictionary learning problems are usually nonconvex and NP-hard, and alternate minimization methods are often computationally expensive. In this article, we propose a method for Fourier domain robust denoising decomposition and adaptive patch MRI reconstruction (DDAPR). DDAPR is a two-step optimization method for MRI reconstruction in the presence of noise and low undersampled data. It includes the low-rank and sparse denoising reconstruction model (LSDRM) and the robust dictionary learning reconstruction model (RDLRM). In the first step, we propose LSDRM for different domains. For the optimization solution, the proximal gradient method is used to optimize LSDRM by singular value decomposition and soft threshold algorithms. In the second step, we propose RDLRM, which is an effective adaptive patch method by introducing a low-rank and sparse penalty adaptive patch dictionary and using a sparse rank-one matrix to approximate the undersampled data. Then, the block coordinate descent (BCD) method is used to optimize the variables. The BCD optimization process involves valid closed-form solutions. Extensive numerical experiments show that the proposed method has a better performance than previous methods in image reconstruction based on compressed sensing or deep learning.

A Convolutional Neural Network-Based Quantization Method for Block Compressed Sensing of Images

Block compressed sensing (BCS) is a promising method for resource-constrained image/video coding applications. However, the quantization of BCS measurements has posed a challenge, leading to significant quantization errors and encoding redundancy. In this paper, we propose a quantization method for BCS measurements using convolutional neural networks (CNN). The quantization process maps measurements to quantized data that follow a uniform distribution based on the measurements' distribution, which aims to maximize the amount of information carried by the quantized data. The dequantization process restores the quantized data to data that conform to the measurements' distribution. The restored data are then modified by the correlation information of the measurements drawn from the quantized data, with the goal of minimizing the quantization errors. The proposed method uses CNNs to construct quantization and dequantization processes, and the networks are trained jointly. The distribution parameters of each block are used as side information, which is quantized with 1 bit by the same method. Extensive experiments on four public datasets showed that, compared with uniform quantization and entropy coding, the proposed method can improve the PSNR by an average of 0.48 dB without using entropy coding when the compression bit rate is 0.1 bpp.

A model-based MR parameter mapping network robust to substantial variations in acquisition settings

Deep learning methods show great potential for the efficient and precise estimation of quantitative parameter maps from multiple magnetic resonance (MR) images. Current deep learning-based MR parameter mapping (MPM) methods are mostly trained and tested using data with specific acquisition settings. However, scan protocols usually vary with centers, scanners, and studies in practice. Thus, deep learning methods applicable to MPM with varying acquisition settings are highly required but still rarely investigated. In this work, we develop a model-based deep network termed MMPM-Net for robust MPM with varying acquisition settings. A deep learning-based denoiser is introduced to construct the regularization term in the nonlinear inversion problem of MPM. The alternating direction method of multipliers is used to solve the optimization problem and then unrolled to construct MMPM-Net. The variation in acquisition parameters can be addressed by the data fidelity component in MMPM-Net. Extensive experiments are performed on R2 mapping and R1 mapping datasets with substantial variations in acquisition settings, and the results demonstrate that the proposed MMPM-Net method outperforms other state-of-the-art MR parameter mapping methods both qualitatively and quantitatively.

Noise-Generating and Imaging Mechanism Inspired Implicit Regularization Learning Network for Low Dose CT Reconstrution

Low-dose computed tomography (LDCT) helps to reduce radiation risks in CT scanning while maintaining image quality, which involves a consistent pursuit of lower incident rays and higher reconstruction performance. Although deep learning approaches have achieved encouraging success in LDCT reconstruction, most of them treat the task as a general inverse problem in either the image domain or the dual (sinogram and image) domains. Such frameworks have not considered the original noise generation of the projection data and suffer from limited performance improvement for the LDCT task. In this paper, we propose a novel reconstruction model based on noise-generating and imaging mechanism in full-domain, which fully considers the statistical properties of intrinsic noises in LDCT and prior information in sinogram and image domains. To solve the model, we propose an optimization algorithm based on the proximal gradient technique. Specifically, we derive the approximate solutions of the integer programming problem on the projection data theoretically. Instead of hand-crafting the sinogram and image regularizers, we propose to unroll the optimization algorithm to be a deep network. The network implicitly learns the proximal operators of sinogram and image regularizers with two deep neural networks, providing a more interpretable and effective reconstruction procedure. Numerical results demonstrate our proposed method improvements of > 2.9 dB in peak signal to noise ratio, > 1.4% promotion in structural similarity metric, and > 9 HU decrements in root mean square error over current state-of-the-art LDCT methods.

Improving deep PROPELLER MRI via synthetic blade augmentation and enhanced generalization

In PROPELLER MRI, obtaining sufficient high-quality blade data remains a challenge, so the efficiency and generalization of deep learning-based reconstruction models are deteriorated. Due to narrow rotated and translated blades acquired in PROPELLER, the technique of data augmentation that is used for deep learning-based Cartesian MRI reconstruction cannot be directly applied. To address the issue, this paper introduces a novel approach for the generation of synthetic PROPELLER blades, and it is subsequently employed in data augmentation for undersampled blades reconstruction. The principal aim of this study is to address the challenges of reconstructing undersampled blades to enhance both image quality and computational efficiency. Evaluation metrics including PSNR, NMSE, and SSIM indicate superior performance of the model trained with augmented data compared to non-augmented counterparts. The synthetic blade augmentation significantly enhances the model's generalization capability and enables robust performance across varying imaging conditions. Furthermore, the study demonstrates the feasibility of utilizing synthetic blades exclusively in the training phase, suggesting a reduced dependency on real PROPELLER blades. This innovation in synthetic blade generation and data augmentation technique contributes to enhanced image quality and improved generalization capability of the associated deep learning model for PROPELLER MRI reconstruction.

Transient Interference Excision and Spectrum Reconstruction with Partial Samples Using Modified Alternating Direction Method of Multipliers-Net for the Over-the-Horizon Radar

Transient interference often submerges the actual targets when employing over-the-horizon radar (OTHR) to detect targets. In addition, modern OTHR needs to carry out multi-target detection from sea to air, resulting in the sparse sampling of echo data. The sparse OTHR signal will raise serious grating lobes using conventional methods and thus degrade target detection performance. This article proposes a modified Alternating Direction Method of Multipliers (ADMM)-Net to reconstruct the target and clutter spectrum of sparse OTHR signals so that target detection can be performed normally. Firstly, transient interferences are identified based on the sparse basis representation and then excised. Therefore, the processed signal can be seen as a sparse OTHR signal. By solving the Doppler sparsity-constrained optimization with the trained network, the complete Doppler spectrum is reconstructed effectively for target detection. Compared with traditional sparse solution methods, the presented approach can balance the efficiency and accuracy of OTHR signal spectrum reconstruction. Both simulation and real-measured OTHR data proved the proposed approach's performance.

Image Reconstruction for Accelerated MR Scan With Faster Fourier Convolutional Neural Networks

High quality image reconstruction from undersampled k -space data is key to accelerating MR scanning. Current deep learning methods are limited by the small receptive fields in reconstruction networks, which restrict the exploitation of long-range information, and impede the mitigation of full-image artifacts, particularly in 3D reconstruction tasks. Additionally, the substantial computational demands of 3D reconstruction considerably hinder advancements in related fields. To tackle these challenges, we propose the following: 1) A novel convolution operator named Faster Fourier Convolution (FasterFC), aims at providing an adaptable broad receptive field for spatial domain reconstruction networks with fast computational speed. 2) A split-slice strategy that substantially reduces the computational load of 3D reconstruction, enabling high-resolution, multi-coil, 3D MR image reconstruction while fully utilizing inter-layer and intra-layer information. 3) A single-to-group algorithm that efficiently utilizes scan-specific and data-driven priors to enhance k -space interpolation effects. 4) A multi-stage, multi-coil, 3D fast MRI method, called the faster Fourier convolution based single-to-group network (FAS-Net), comprising a single-to-group k -space interpolation algorithm and a FasterFC-based image domain reconstruction module, significantly minimizes the computational demands of 3D reconstruction through split-slice strategy. Experimental evaluations conducted on the NYU fastMRI and Stanford MRI Data datasets reveal that the FasterFC significantly enhances the quality of both 2D and 3D reconstruction results. Moreover, FAS-Net, characterized as a method that can achieve high-resolution (320, 320, 256), multi-coil, (8 coils), 3D fast MRI, exhibits superior reconstruction performance compared to other state-of-the-art 2D and 3D methods.

MRI motion artifact reduction using a conditional diffusion probabilistic model (MAR-CDPM)

BACKGROUND

High-resolution magnetic resonance imaging (MRI) with excellent soft-tissue contrast is a valuable tool utilized for diagnosis and prognosis. However, MRI sequences with long acquisition time are susceptible to motion artifacts, which can adversely affect the accuracy of post-processing algorithms.

PURPOSE

This study proposes a novel retrospective motion correction method named "motion artifact reduction using conditional diffusion probabilistic model" (MAR-CDPM). The MAR-CDPM aimed to remove motion artifacts from multicenter three-dimensional contrast-enhanced T1 magnetization-prepared rapid acquisition gradient echo (3D ceT1 MPRAGE) brain dataset with different brain tumor types.

MATERIALS AND METHODS

This study employed two publicly accessible MRI datasets: one containing 3D ceT1 MPRAGE and 2D T2-fluid attenuated inversion recovery (FLAIR) images from 230 patients with diverse brain tumors, and the other comprising 3D T1-weighted (T1W) MRI images of 148 healthy volunteers, which included real motion artifacts. The former was used to train and evaluate the model using the in silico data, and the latter was used to evaluate the model performance to remove real motion artifacts. A motion simulation was performed in k-space domain to generate an in silico dataset with minor, moderate, and heavy distortion levels. The diffusion process of the MAR-CDPM was then implemented in k-space to convert structure data into Gaussian noise by gradually increasing motion artifact levels. A conditional network with a Unet backbone was trained to reverse the diffusion process to convert the distorted images to structured data. The MAR-CDPM was trained in two scenarios: one conditioning on the time step t $t$ of the diffusion process, and the other conditioning on both t $t$ and T2-FLAIR images. The MAR-CDPM was quantitatively and qualitatively compared with supervised Unet, Unet conditioned on T2-FLAIR, CycleGAN, Pix2pix, and Pix2pix conditioned on T2-FLAIR models. To quantify the spatial distortions and the level of remaining motion artifacts after applying the models, quantitative metrics were reported including normalized mean squared error (NMSE), structural similarity index (SSIM), multiscale structural similarity index (MS-SSIM), peak signal-to-noise ratio (PSNR), visual information fidelity (VIF), and multiscale gradient magnitude similarity deviation (MS-GMSD). Tukey's Honestly Significant Difference multiple comparison test was employed to quantify the difference between the models where p-value < 0.05 $ < 0.05$ was considered statistically significant.

RESULTS

Qualitatively, MAR-CDPM outperformed these methods in preserving soft-tissue contrast and different brain regions. It also successfully preserved tumor boundaries for heavy motion artifacts, like the supervised method. Our MAR-CDPM recovered motion-free in silico images with the highest PSNR and VIF for all distortion levels where the differences were statistically significant (p-values< 0.05 $< 0.05$ ). In addition, our method conditioned on t and T2-FLAIR outperformed (p-values< 0.05 $< 0.05$ ) other methods to remove motion artifacts from the in silico dataset in terms of NMSE, MS-SSIM, SSIM, and MS-GMSD. Moreover, our method conditioned on only t outperformed generative models (p-values< 0.05 $< 0.05$ ) and had comparable performances compared with the supervised model (p-values> 0.05 $> 0.05$ ) to remove real motion artifacts.

CONCLUSIONS

The MAR-CDPM could successfully remove motion artifacts from 3D ceT1 MPRAGE. It is particularly beneficial for elderly who may experience involuntary movements during high-resolution MRI imaging with long acquisition times.

CTV-Net: Complex-Valued TV-Driven Network With Nested Topology for 3-D SAR Imaging

The regularization-based approaches offer promise in improving synthetic aperture radar (SAR) imaging quality while reducing system complexity. However, the widely applied l1 regularization model is hindered by their hypothesis of inherent sparsity, causing unreal estimations of surface-like targets. Inspired by the edge-preserving property of total variation (TV), we propose a new complex-valued TV (CTV)-driven interpretable neural network with nested topology, i.e., CTV-Net, for 3-D SAR imaging. In our scheme, based on the 2-D holography imaging operator, the CTV-driven optimization model is constructed to pursue precise estimations in weakly sparse scenarios. Subsequently, a nested algorithmic framework, i.e., complex-valued TV-driven fast iterative shrinkage thresholding (CTV-FIST), is derived from the theory of proximal gradient descent (PGD) and FIST algorithm, theoretically supporting the design of CTV-Net. In CTV-Net, the trainable weights are layer-varied and functionally relevant to the hyperparameters of CTV-FIST, which aims to constrain the algorithmic parameters to update in a well-conditioned tendency. All weights are learned by end-to-end training based on a two-term cost function, which bounds the measurement fidelity and TV norm simultaneously. Under the guidance of the SAR signal model, a reasonably sized training set is generated, by randomly selecting reference images from the MNIST set and consequently synthesizing complex-valued label signals. Finally, the methodology is validated, numerically and visually, by extensive SAR simulations and real-measured experiments, and the results demonstrate the viability and efficiency of the proposed CTV-Net in the cases of recovering 3-D SAR images from incomplete echoes.

Nonlocal based FISTA network for noninvasive cardiac transmembrane potential imaging

Objective. The primary aim of our study is to advance our understanding and diagnosis of cardiac diseases. We focus on the reconstruction of myocardial transmembrane potential (TMP) from body surface potential mapping.Approach. We introduce a novel methodology for the reconstruction of the dynamic distribution of TMP. This is achieved through the integration of convolutional neural networks with conventional optimization algorithms. Specifically, we utilize the subject-specific transfer matrix to describe the dynamic changes in TMP distribution and ECG observations at the body surface. To estimate the TMP distribution, we employ LNFISTA-Net, a learnable non-local regularized iterative shrinkage-thresholding network. The coupled estimation processes are iteratively repeated until convergence.Main results. Our experiments demonstrate the capabilities and benefits of this strategy. The results highlight the effectiveness of our approach in accurately estimating the TMP distribution, thereby providing a reliable method for the diagnosis of cardiac diseases.Significance. Our approach demonstrates promising results, highlighting its potential utility for a range of applications in the medical field. By providing a more accurate and dynamic reconstruction of TMP, our methodology could significantly improve the diagnosis and treatment of cardiac diseases, thereby contributing to advancements in healthcare.

Multi-Contrast Complementary Learning for Accelerated MR Imaging

Thanks to its powerful ability to depict high-resolution anatomical information, magnetic resonance imaging (MRI) has become an essential non-invasive scanning technique in clinical practice. However, excessive acquisition time often leads to the degradation of image quality and psychological discomfort among subjects, hindering its further popularization. Besides reconstructing images from the undersampled protocol itself, multi-contrast MRI protocols bring promising solutions by leveraging additional morphological priors for the target modality. Nevertheless, previous multi-contrast techniques mainly adopt a simple fusion mechanism that inevitably ignores valuable knowledge. In this work, we propose a novel multi-contrast complementary information aggregation network named MCCA, aiming to exploit available complementary representations fully to reconstruct the undersampled modality. Specifically, a multi-scale feature fusion mechanism has been introduced to incorporate complementary-transferable knowledge into the target modality. Moreover, a hybrid convolution transformer block was developed to extract global-local context dependencies simultaneously, which combines the advantages of CNNs while maintaining the merits of Transformers. Compared to existing MRI reconstruction methods, the proposed method has demonstrated its superiority through extensive experiments on different datasets under different acceleration factors and undersampling patterns.

Cooperative-Net: An end-to-end multi-task interaction network for unified reconstruction and segmentation of MR image

BACKGROUND AND OBJECTIVE

In clinical applications, there is an increasing demand for rapid acquisition and automated analysis of magnetic resonance imaging (MRI) data. However, most existing methods focus on either MR image reconstruction from undersampled data or segmentation using fully sampled data, hardly considering MR image segmentation in fast imaging scenarios. Consequently, it is imperative to investigate a multi-task approach that can simultaneously achieve high scanning acceleration and accurate segmentation results.

METHODS

In this paper, we propose a novel end-to-end multi-task interaction network, termed as the Cooperative-Net, which integrates accelerated MR imaging and multi-class tissue segmentation into a unified framework. The Cooperative-Net consists of alternating reconstruction modules and segmentation modules. To facilitate effective interaction between the two tasks, we introduce the spatial-adaptive semantic guidance module, which leverages the semantic map as a structural prior to guide MR image reconstruction. Furthermore, we propose a novel unrolling network with a multi-path shrinkage structure for MR image reconstruction. This network consists of parallel learnable shrinkage paths to handle varying degrees of degradation across different frequency components in the undersampled MR image, effectively improving the quality of the recovered image.

RESULTS

We use two publicly available datasets, including the cardiac and knee MR datasets, to validate the efficacy of our proposed Cooperative-Net. Through qualitative and quantitative analysis, we demonstrate that our method outperforms existing state-of-the-art multi-task approaches for joint MR image reconstruction and segmentation.

CONCLUSIONS

The proposed Cooperative-Net is capable of achieving both high accelerated MR imaging and accurate multi-class tissue segmentation.

Annihilation-Net: Learned annihilation relation for dynamic MR imaging

BACKGROUND

Deep learning methods driven by the low-rank regularization have achieved attractive performance in dynamic magnetic resonance (MR) imaging. The effectiveness of existing methods lies mainly in their ability to capture interframe relationships using network modules, which are lack interpretability.

PURPOSE

This study aims to design an interpretable methodology for modeling interframe relationships using convolutiona networks, namely Annihilation-Net and use it for accelerating dynamic MRI.

METHODS

Based on the equivalence between Hankel matrix product and convolution, we utilize convolutional networks to learn the null space transform for characterizing low-rankness. We employ low-rankness to represent interframe correlations in dynamic MR imaging, while combining with sparse constraints in the compressed sensing framework. The corresponding optimization problem is solved in an iterative form with the semi-quadratic splitting method (HQS). The iterative steps are unrolled into a network, dubbed Annihilation-Net. All the regularization parameters and null space transforms are set as learnable in the Annihilation-Net.

RESULTS

Experiments on the cardiac cine dataset show that the proposed model outperforms other competing methods both quantitatively and qualitatively. The training set and test set have 800 and 118 images, respectively.

CONCLUSIONS

The proposed Annihilation-Net improves the reconstruction quality of accelerated dynamic MRI with better interpretability.

RPCA-AENet: Clutter Suppression and Simultaneous Stationary Scene and Moving Targets Imaging in the Presence of Motion Errors

Clutter suppression and ground moving target imaging in synthetic aperture radar (SAR) system have been receiving increasing attention for both civilian and military applications. The problem of clutter suppression and ground moving target imaging in practical applications is much more challenging due to the motion error of the radar platform. In this article, we focus on the problems of clutter suppression and simultaneous stationary and moving target imaging in the presence of motion errors. Specifically, we propose a robust principal component analysis autoencoder network (RPCA-AENet) in a single-channel SAR system. In RPCA-AENet, the encoder transforms the SAR echo into imaging results of stationary scene and ground moving targets, and the decoder regenerates the SAR echo using the obtained imaging results. The encoder is designed by the unfolded robust principal component analysis (RPCA), while the decoder is formulated into two dense layers and one additional layer. Joint reconstruction loss, entropy loss, and measurement distance loss are utilized to guide the training of the RPCA-AENet. Notably, the algorithm operates in a totally self-supervised form and requires no other labeled SAR data. The methodology was tested on numerical SAR data. These tests show that the proposed architecture outperforms other state-of-the-art methods.

Promoting fast MR imaging pipeline by full-stack AI

Magnetic resonance imaging (MRI) is a widely used imaging modality in clinics for medical disease diagnosis, staging, and follow-up. Deep learning has been extensively used to accelerate k-space data acquisition, enhance MR image reconstruction, and automate tissue segmentation. However, these three tasks are usually treated as independent tasks and optimized for evaluation by radiologists, thus ignoring the strong dependencies among them; this may be suboptimal for downstream intelligent processing. Here, we present a novel paradigm, full-stack learning (FSL), which can simultaneously solve these three tasks by considering the overall imaging process and leverage the strong dependence among them to further improve each task, significantly boosting the efficiency and efficacy of practical MRI workflows. Experimental results obtained on multiple open MR datasets validate the superiority of FSL over existing state-of-the-art methods on each task. FSL has great potential to optimize the practical workflow of MRI for medical diagnosis and radiotherapy.

Hierarchical neural architecture search with adaptive global-local feature learning for Magnetic Resonance Image reconstruction

Neural architecture search (NAS) has been introduced into the design of deep neural network architectures for Magnetic Resonance Imaging (MRI) reconstruction since NAS-based methods can acquire the complex network architecture automatically without professional designing experience and improve the model's generalization ability. However, current NAS-based MRI reconstruction methods suffer from a lack of efficient operators in the search space, which leads to challenges in effectively recovering high-frequency details. This limitation is primarily due to the prevalent use of convolution operators in the current search space, which struggle to capture both global and local features of MR images simultaneously, resulting in insufficient information utilization. To address this issue, a generative adversarial network (GAN) based model is proposed to reconstruct the MR image from under-sampled K-space data. Firstly, parameterized global and local feature learning modules at multiple scales are added into the search space to improve the capability of recovering high-frequency details. Secondly, to mitigate the increased search time caused by the augmented search space, a hierarchical NAS is designed to learn the global-local feature learning modules that enable the reconstruction network to learn global and local information of MR images at different scales adaptively. Thirdly, to reduce the number of network parameters and computational complexity, the standard operations in global-local feature learning modules are replaced with lightweight operations. Finally, experiments on several publicly available brain MRI image datasets evaluate the performance of the proposed method. Compared to the state-of-the-art MRI reconstruction methods, the proposed method yields better reconstruction results in terms of peak signal-to-noise ratio and structural similarity at a lower computational cost. Additionally, our reconstruction results are validated through a brain tumor classification task, affirming the practicability of the proposed method. Our code is available at https://github.com/wwHwo/HNASMRI.

Deep Generalized Learning Model for PET Image Reconstruction

Low-count positron emission tomography (PET) imaging is challenging because of the ill-posedness of this inverse problem. Previous studies have demonstrated that deep learning (DL) holds promise for achieving improved low-count PET image quality. However, almost all data-driven DL methods suffer from fine structure degradation and blurring effects after denoising. Incorporating DL into the traditional iterative optimization model can effectively improve its image quality and recover fine structures, but little research has considered the full relaxation of the model, resulting in the performance of this hybrid model not being sufficiently exploited. In this paper, we propose a learning framework that deeply integrates DL and an alternating direction of multipliers method (ADMM)-based iterative optimization model. The innovative feature of this method is that we break the inherent forms of the fidelity operators and use neural networks to process them. The regularization term is deeply generalized. The proposed method is evaluated on simulated data and real data. Both the qualitative and quantitative results show that our proposed neural network method can outperform partial operator expansion-based neural network methods, neural network denoising methods and traditional methods.

AliasNet: Alias artefact suppression network for accelerated phase-encode MRI

Sparse reconstruction is an important aspect of MRI, helping to reduce acquisition time and improve spatial-temporal resolution. Popular methods are based mostly on compressed sensing (CS), which relies on the random sampling of k-space to produce incoherent (noise-like) artefacts. Due to hardware constraints, 1D Cartesian phase-encode under-sampling schemes are popular for 2D CS-MRI. However, 1D under-sampling limits 2D incoherence between measurements, yielding structured aliasing artefacts (ghosts) that may be difficult to remove assuming a 2D sparsity model. Reconstruction algorithms typically deploy direction-insensitive 2D regularisation for these direction-associated artefacts. Recognising that phase-encode artefacts can be separated into contiguous 1D signals, we develop two decoupling techniques that enable explicit 1D regularisation and leverage the excellent 1D incoherence characteristics. We also derive a combined 1D + 2D reconstruction technique that takes advantage of spatial relationships within the image. Experiments conducted on retrospectively under-sampled brain and knee data demonstrate that combination of the proposed 1D AliasNet modules with existing 2D deep learned (DL) recovery techniques leads to an improvement in image quality. We also find AliasNet enables a superior scaling of performance compared to increasing the size of the original 2D network layers. AliasNet therefore improves the regularisation of aliasing artefacts arising from phase-encode under-sampling, by tailoring the network architecture to account for their expected appearance. The proposed 1D + 2D approach is compatible with any existing 2D DL recovery technique deployed for this application.