ADMM-CSNet: A Deep Learning Approach for Image Compressive Sensing

Deep Generalized Learning Model for PET Image Reconstruction

Low-count positron emission tomography (PET) imaging is challenging because of the ill-posedness of this inverse problem. Previous studies have demonstrated that deep learning (DL) holds promise for achieving improved low-count PET image quality. However, almost all data-driven DL methods suffer from fine structure degradation and blurring effects after denoising. Incorporating DL into the traditional iterative optimization model can effectively improve its image quality and recover fine structures, but little research has considered the full relaxation of the model, resulting in the performance of this hybrid model not being sufficiently exploited. In this paper, we propose a learning framework that deeply integrates DL and an alternating direction of multipliers method (ADMM)-based iterative optimization model. The innovative feature of this method is that we break the inherent forms of the fidelity operators and use neural networks to process them. The regularization term is deeply generalized. The proposed method is evaluated on simulated data and real data. Both the qualitative and quantitative results show that our proposed neural network method can outperform partial operator expansion-based neural network methods, neural network denoising methods and traditional methods.

AliasNet: Alias artefact suppression network for accelerated phase-encode MRI

Sparse reconstruction is an important aspect of MRI, helping to reduce acquisition time and improve spatial-temporal resolution. Popular methods are based mostly on compressed sensing (CS), which relies on the random sampling of k-space to produce incoherent (noise-like) artefacts. Due to hardware constraints, 1D Cartesian phase-encode under-sampling schemes are popular for 2D CS-MRI. However, 1D under-sampling limits 2D incoherence between measurements, yielding structured aliasing artefacts (ghosts) that may be difficult to remove assuming a 2D sparsity model. Reconstruction algorithms typically deploy direction-insensitive 2D regularisation for these direction-associated artefacts. Recognising that phase-encode artefacts can be separated into contiguous 1D signals, we develop two decoupling techniques that enable explicit 1D regularisation and leverage the excellent 1D incoherence characteristics. We also derive a combined 1D + 2D reconstruction technique that takes advantage of spatial relationships within the image. Experiments conducted on retrospectively under-sampled brain and knee data demonstrate that combination of the proposed 1D AliasNet modules with existing 2D deep learned (DL) recovery techniques leads to an improvement in image quality. We also find AliasNet enables a superior scaling of performance compared to increasing the size of the original 2D network layers. AliasNet therefore improves the regularisation of aliasing artefacts arising from phase-encode under-sampling, by tailoring the network architecture to account for their expected appearance. The proposed 1D + 2D approach is compatible with any existing 2D DL recovery technique deployed for this application.

Joint Cross-Attention Network With Deep Modality Prior for Fast MRI Reconstruction

Current deep learning-based reconstruction models for accelerated multi-coil magnetic resonance imaging (MRI) mainly focus on subsampled k-space data of single modality using convolutional neural network (CNN). Although dual-domain information and data consistency constraint are commonly adopted in fast MRI reconstruction, the performance of existing models is still limited mainly by three factors: inaccurate estimation of coil sensitivity, inadequate utilization of structural prior, and inductive bias of CNN. To tackle these challenges, we propose an unrolling-based joint Cross-Attention Network, dubbed as jCAN, using deep guidance of the already acquired intra-subject data. Particularly, to improve the performance of coil sensitivity estimation, we simultaneously optimize the latent MR image and sensitivity map (SM). Besides, we introduce Gating layer and Gaussian layer into SM estimation to alleviate the "defocus" and "over-coupling" effects and further ameliorate the SM estimation. To enhance the representation ability of the proposed model, we deploy Vision Transformer (ViT) and CNN in the image and k-space domains, respectively. Moreover, we exploit pre-acquired intra-subject scan as reference modality to guide the reconstruction of subsampled target modality by resorting to the self- and cross-attention scheme. Experimental results on public knee and in-house brain datasets demonstrate that the proposed jCAN outperforms the state-of-the-art methods by a large margin in terms of SSIM and PSNR for different acceleration factors and sampling masks. Our code is publicly available at https://github.com/sunkg/jCAN.

DERnet: a deep neural network for end-to-end reconstruction in magnetic particle imaging

Objective. Magnetic particle imaging (MPI) shows potential for contributing to biomedical research and clinical practice. However, MPI images are effectively affected by noise in the signal as its reconstruction is an ill-posed inverse problem. Thus, effective reconstruction method is required to reduce the impact of the noise while mapping signals to MPI images. Traditional methods rely on the hand-crafted data-consistency (DC) term and regularization term based on spatial priors to achieve noise-reducing and reconstruction. While these methods alleviate the ill-posedness and reduce noise effects, they may be difficult to fully capture spatial features.Approach. In this study, we propose a deep neural network for end-to-end reconstruction (DERnet) in MPI that emulates the DC term and regularization term using the feature mapping subnetwork and post-processing subnetwork, respectively, but in a data-driven manner. By doing so, DERnet can better capture signal and spatial features without relying on hand-crafted priors and strategies, thereby effectively reducing noise interference and achieving superior reconstruction quality.Main results. Our data-driven method outperforms the state-of-the-art algorithms with an improvement of 0.9-8.8 dB in terms of peak signal-to-noise ratio under various noise levels. The result demonstrates the advantages of our approach in suppressing noise interference. Furthermore, DERnet can be employed for measured data reconstruction with improved fidelity and reduced noise. In conclusion, our proposed method offers performance benefits in reducing noise interference and enhancing reconstruction quality by effectively capturing signal and spatial features.Significance. DERnet is a promising candidate method to improve MPI reconstruction performance and facilitate its more in-depth biomedical application.

A deep unrolled neural network for real-time MRI-guided brain intervention

Accurate navigation and targeting are critical for neurological interventions including biopsy and deep brain stimulation. Real-time image guidance further improves surgical planning and MRI is ideally suited for both pre- and intra-operative imaging. However, balancing spatial and temporal resolution is a major challenge for real-time interventional MRI (i-MRI). Here, we proposed a deep unrolled neural network, dubbed as LSFP-Net, for real-time i-MRI reconstruction. By integrating LSFP-Net and a custom-designed, MR-compatible interventional device into a 3 T MRI scanner, a real-time MRI-guided brain intervention system is proposed. The performance of the system was evaluated using phantom and cadaver studies. 2D/3D real-time i-MRI was achieved with temporal resolutions of 80/732.8 ms, latencies of 0.4/3.66 s including data communication, processing and reconstruction time, and in-plane spatial resolution of 1 × 1 mm2. The results demonstrated that the proposed method enables real-time monitoring of the remote-controlled brain intervention, and showed the potential to be readily integrated into diagnostic scanners for image-guided neurosurgery.

A Real-Time and Robust Neural Network Model for Low-Measurement-Rate Compressed-Sensing Image Reconstruction

Compressed sensing (CS) is a popular data compression theory for many computer vision tasks, but the high reconstruction complexity for images prevents it from being used in many real-world applications. Existing end-to-end learning methods achieved real time sensing but lack theory guarantee for robust reconstruction results. This paper proposes a neural network called RootsNet, which integrates the CS mechanism into the network to prevent error propagation. So, RootsNet knows what will happen if some modules in the network go wrong. It also implements real-time and successfully reconstructed extremely low measurement rates that are impossible for traditional optimization-theory-based methods. For qualitative validation, RootsNet is implemented in two real-world measurement applications, i.e., a near-field microwave imaging system and a pipeline inspection system, where RootsNet easily saves 60% more measurement time and 95% more data compared with the state-of-the-art optimization-theory-based reconstruction methods. Without losing generality, comprehensive experiments are performed on general datasets, including evaluating the key components in RootsNet, the reconstruction uncertainty, quality, and efficiency. RootsNet has the best uncertainty performance and efficiency, and achieves the best reconstruction quality under super low-measurement rates.

Nanoscale chemical imaging with structured X-ray illumination

High-resolution imaging with compositional and chemical sensitivity is crucial for a wide range of scientific and engineering disciplines. Although synchrotron X-ray imaging through spectromicroscopy has been tremendously successful and broadly applied, it encounters challenges in achieving enhanced detection sensitivity, satisfactory spatial resolution, and high experimental throughput simultaneously. In this work, based on structured illumination, we develop a single-pixel X-ray imaging approach coupled with a generative image reconstruction model for mapping the compositional heterogeneity with nanoscale resolvability. This method integrates a full-field transmission X-ray microscope with an X-ray fluorescence detector and eliminates the need for nanoscale X-ray focusing and raster scanning. We experimentally demonstrate the effectiveness of our approach by imaging a battery sample composed of mixed cathode materials and successfully retrieving the compositional variations of the imaged cathode particles. Bridging the gap between structural and chemical characterizations using X-rays, this technique opens up vast opportunities in the fields of biology, environmental, and materials science, especially for radiation-sensitive samples.

Parallel-stream fusion of scan-specific and scan-general priors for learning deep MRI reconstruction in low-data regimes

Magnetic resonance imaging (MRI) is an essential diagnostic tool that suffers from prolonged scan times. Reconstruction methods can alleviate this limitation by recovering clinically usable images from accelerated acquisitions. In particular, learning-based methods promise performance leaps by employing deep neural networks as data-driven priors. A powerful approach uses scan-specific (SS) priors that leverage information regarding the underlying physical signal model for reconstruction. SS priors are learned on each individual test scan without the need for a training dataset, albeit they suffer from computationally burdening inference with nonlinear networks. An alternative approach uses scan-general (SG) priors that instead leverage information regarding the latent features of MRI images for reconstruction. SG priors are frozen at test time for efficiency, albeit they require learning from a large training dataset. Here, we introduce a novel parallel-stream fusion model (PSFNet) that synergistically fuses SS and SG priors for performant MRI reconstruction in low-data regimes, while maintaining competitive inference times to SG methods. PSFNet implements its SG prior based on a nonlinear network, yet it forms its SS prior based on a linear network to maintain efficiency. A pervasive framework for combining multiple priors in MRI reconstruction is algorithmic unrolling that uses serially alternated projections, causing error propagation under low-data regimes. To alleviate error propagation, PSFNet combines its SS and SG priors via a novel parallel-stream architecture with learnable fusion parameters. Demonstrations are performed on multi-coil brain MRI for varying amounts of training data. PSFNet outperforms SG methods in low-data regimes, and surpasses SS methods with few tens of training samples. On average across tasks, PSFNet achieves 3.1 dB higher PSNR, 2.8% higher SSIM, and 0.3 × lower RMSE than baselines. Furthermore, in both supervised and unsupervised setups, PSFNet requires an order of magnitude lower samples compared to SG methods, and enables an order of magnitude faster inference compared to SS methods. Thus, the proposed model improves deep MRI reconstruction with elevated learning and computational efficiency.

DC-SiamNet: Deep contrastive Siamese network for self-supervised MRI reconstruction

Reconstruction methods based on deep learning have greatly shortened the data acquisition time of magnetic resonance imaging (MRI). However, these methods typically utilize massive fully sampled data for supervised training, restricting their application in certain clinical scenarios and posing challenges to the reconstruction effect when high-quality MR images are unavailable. Recently, self-supervised methods have been developed that only undersampled MRI images participate in the network training. Nevertheless, due to the lack of complete referable MR image data, self-supervised reconstruction is prone to produce incorrect structure contents, such as unnatural texture details and over-smoothed tissue sites. To solve this problem, we propose a self-supervised Deep Contrastive Siamese Network (DC-SiamNet) for fast MR imaging. First, DC-SiamNet performs the reconstruction with a Siamese unrolled structure and obtains visual representations in different iterative phases. Particularly, an attention-weighted average pooling module is employed at the bottleneck layer of the U-shape regularization unit, which can effectively aggregate valuable local information of the underlying feature map in the generated representation vector. Then, a novel hybrid loss function is designed to drive the self-supervised reconstruction and contrastive learning simultaneously by forcing the output consistency across different branches in the frequency domain, the image domain, and the latent space. The proposed method is extensively evaluated with different sampling patterns on the IXI brain dataset and the MRINet knee dataset. Experimental results show that DC-SiamNet can achieve 0.93 in structural similarity and 33.984 dB in peak signal-to-noise ratio on the IXI brain dataset under 8x acceleration. It has better reconstruction accuracy than other methods, and the performance is close to the corresponding model trained with full supervision, especially when the sampling rate is low. In addition, generalization experiments verify that our method has a strong cross-domain reconstruction ability for different contrast brain images.

Hierarchical Optimization-Derived Learning

In recent years, by utilizing optimization techniques to formulate the propagation of deep model, a variety of so-called Optimization-Derived Learning (ODL) approaches have been proposed to address diverse learning and vision tasks. Although having achieved relatively satisfying practical performance, there still exist fundamental issues in existing ODL methods. In particular, current ODL methods tend to consider model constructing and learning as two separate phases, and thus fail to formulate their underlying coupling and depending relationship. In this work, we first establish a new framework, named Hierarchical ODL (HODL), to simultaneously investigate the intrinsic behaviors of optimization-derived model construction and its corresponding learning process. Then we rigorously prove the joint convergence of these two sub-tasks, from the perspectives of both approximation quality and stationary analysis. To our best knowledge, this is the first theoretical guarantee for these two coupled ODL components: optimization and learning. We further demonstrate the flexibility of our framework by applying HODL to challenging learning tasks, which have not been properly addressed by existing ODL methods. Finally, we conduct extensive experiments on both synthetic data and real applications in vision and other learning tasks to verify the theoretical properties and practical performance of HODL in various application scenarios.

Multi-Channel Representation Learning Enhanced Unfolding Multi-Scale Compressed Sensing Network for High Quality Image Reconstruction

Deep Unfolding Networks (DUNs) serve as a predominant approach for Compressed Sensing (CS) reconstruction algorithms by harnessing optimization. However, a notable constraint within the DUN framework is the restriction to single-channel inputs and outputs at each stage during gradient descent computations. This constraint compels the feature maps of the proximal mapping module to undergo multi-channel to single-channel dimensionality reduction, resulting in limited feature characterization capabilities. Furthermore, most prevalent reconstruction networks rely on single-scale structures, neglecting the extraction of features from different scales, thereby impeding the overall reconstruction network's performance. To address these limitations, this paper introduces a novel CS reconstruction network termed the Multi-channel and Multi-scale Unfolding Network (MMU-Net). MMU-Net embraces a multi-channel approach, featuring the incorporation of Adap-SKConv with an attention mechanism to facilitate the exchange of information between gradient terms and enhance the feature map's characterization capacity. Moreover, a Multi-scale Block is introduced to extract multi-scale features, bolstering the network's ability to characterize and reconstruct the images. Our study extensively evaluates MMU-Net's performance across multiple benchmark datasets, including Urban100, Set11, BSD68, and the UC Merced Land Use Dataset, encompassing both natural and remote sensing images. The results of our study underscore the superior performance of MMU-Net in comparison to existing state-of-the-art CS methods.

Low-rank tensor assisted K-space generative model for parallel imaging reconstruction

Although recent deep learning methods, especially generative models, have shown good performance in magnetic resonance imaging, there is still much room for improvement. Considering that the sample number and internal dimension in score-based generative model have key influence on estimating the gradients of data distribution, we present a new idea for parallel imaging reconstruction, named low-rank tensor assisted k-space generative model (LR-KGM). It means that we transform low-rank information into high-dimensional prior information for learning. More specifically, the multi-channel data is constructed into a large Hankel matrix to reduce the number of training samples, which is subsequently collapsed into a tensor for the stage of prior learning. In the testing phase, the low-rank rotation strategy is utilized to impose low-rank constraints on the output tensors of the generative network. Furthermore, we alternate the reconstruction between traditional generative iterations and low-rank high-dimensional tensor iterations. Experimental comparisons with the state-of-the-arts demonstrated that the proposed LR-KGM method achieved better performance.

MMV-Net: A Multiple Measurement Vector Network for Multifrequency Electrical Impedance Tomography

Multifrequency electrical impedance tomography (mfEIT) is an emerging biomedical imaging modality to reveal frequency-dependent conductivity distributions in biomedical applications. Conventional model-based image reconstruction methods suffer from low spatial resolution, unconstrained frequency correlation, and high computational cost. Deep learning has been extensively applied in solving the EIT inverse problem in biomedical and industrial process imaging. However, most existing learning-based approaches deal with the single-frequency setup, which is inefficient and ineffective when extended to the multifrequency setup. This article presents a multiple measurement vector (MMV) model-based learning algorithm named MMV-Net to solve the mfEIT image reconstruction problem. MMV-Net considers the correlations between mfEIT images and unfolds the update steps of the Alternating Direction Method of Multipliers for the MMV problem (MMV-ADMM). The nonlinear shrinkage operator associated with the weighted l2,1 regularization term of MMV-ADMM is generalized in MMV-Net with a cascade of a Spatial Self-Attention module and a Convolutional Long Short-Term Memory (ConvLSTM) module to better capture intrafrequency and interfrequency dependencies. The proposed MMV-Net was validated on our Edinburgh mfEIT Dataset and a series of comprehensive experiments. The results show superior image quality, convergence performance, noise robustness, and computational efficiency against the conventional MMV-ADMM and the state-of-the-art deep learning methods.

A Sparse Model-Inspired Deep Thresholding Network for Exponential Signal Reconstruction-Application in Fast Biological Spectroscopy

The nonuniform sampling (NUS) is a powerful approach to enable fast acquisition but requires sophisticated reconstruction algorithms. Faithful reconstruction from partially sampled exponentials is highly expected in general signal processing and many applications. Deep learning (DL) has shown astonishing potential in this field, but many existing problems, such as lack of robustness and explainability, greatly limit its applications. In this work, by combining the merits of the sparse model-based optimization method and data-driven DL, we propose a DL architecture for spectra reconstruction from undersampled data, called MoDern. It follows the iterative reconstruction in solving a sparse model to build the neural network, and we elaborately design a learnable soft-thresholding to adaptively eliminate the spectrum artifacts introduced by undersampling. Extensive results on both synthetic and biological data show that MoDern enables more robust, high-fidelity, and ultrafast reconstruction than the state-of-the-art methods. Remarkably, MoDern has a small number of network parameters and is trained on solely synthetic data while generalizing well to biological data in various scenarios. Furthermore, we extend it to an open-access and easy-to-use cloud computing platform (XCloud-MoDern), contributing a promising strategy for further development of biological applications.

Variational Data-Free Knowledge Distillation for Continual Learning

Deep neural networks suffer from catastrophic forgetting when trained on sequential tasks in continual learning. Various methods rely on storing data of previous tasks to mitigate catastrophic forgetting, which is prohibited in real-world applications considering privacy and security issues. In this paper, we consider a realistic setting of continual learning, where training data of previous tasks are unavailable and memory resources are limited. We contribute a novel knowledge distillation-based method in an information-theoretic framework by maximizing mutual information between outputs of previously learned and current networks. Due to the intractability of computation of mutual information, we instead maximize its variational lower bound, where the covariance of variational distribution is modeled by a graph convolutional network. The inaccessibility of data of previous tasks is tackled by Taylor expansion, yielding a novel regularizer in network training loss for continual learning. The regularizer relies on compressed gradients of network parameters. It avoids storing previous task data and previously learned networks. Additionally, we employ self-supervised learning technique for learning effective features, which improves the performance of continual learning. We conduct extensive experiments including image classification and semantic segmentation, and the results show that our method achieves state-of-the-art performance on continual learning benchmarks.

Joint Image Reconstruction and Super-Resolution for Accelerated Magnetic Resonance Imaging

Magnetic resonance (MR) image reconstruction and super-resolution are two prominent techniques to restore high-quality images from undersampled or low-resolution k-space data to accelerate MR imaging. Combining undersampled and low-resolution acquisition can further improve the acceleration factor. Existing methods often treat the techniques of image reconstruction and super-resolution separately or combine them sequentially for image recovery, which can result in error propagation and suboptimal results. In this work, we propose a novel framework for joint image reconstruction and super-resolution, aiming to efficiently image recovery and enable fast imaging. Specifically, we designed a framework with a reconstruction module and a super-resolution module to formulate multi-task learning. The reconstruction module utilizes a model-based optimization approach, ensuring data fidelity with the acquired k-space data. Moreover, a deep spatial feature transform is employed to enhance the information transition between the two modules, facilitating better integration of image reconstruction and super-resolution. Experimental evaluations on two datasets demonstrate that our proposed method can provide superior performance both quantitatively and qualitatively.

PARCEL: Physics-Based Unsupervised Contrastive Representation Learning for Multi-Coil MR Imaging

With the successful application of deep learning to magnetic resonance (MR) imaging, parallel imaging techniques based on neural networks have attracted wide attention. However, in the absence of high-quality, fully sampled datasets for training, the performance of these methods is limited. And the interpretability of models is not strong enough. To tackle this issue, this paper proposes a Physics-bAsed unsupeRvised Contrastive rEpresentation Learning (PARCEL) method to speed up parallel MR imaging. Specifically, PARCEL has a parallel framework to contrastively learn two branches of model-based unrolling networks from augmented undersampled multi-coil k-space data. A sophisticated co-training loss with three essential components has been designed to guide the two networks in capturing the inherent features and representations for MR images. And the final MR image is reconstructed with the trained contrastive networks. PARCEL was evaluated on two vivo datasets and compared to five state-of-the-art methods. The results show that PARCEL is able to learn essential representations for accurate MR reconstruction without relying on fully sampled datasets. The code will be made available at https://github.com/ternencewu123/PARCEL.

CAIR: Combining integrated attention with iterative optimization learning for sparse-view CT reconstruction

Sparse-view CT is an efficient way for low dose scanning but degrades image quality. Inspired by the successful use of non-local attention in natural image denoising and compression artifact removal, we proposed a network combining integrated attention and iterative optimization learning for sparse-view CT reconstruction (CAIR). Specifically, we first unrolled the proximal gradient descent into a deep network and added an enhanced initializer between the gradient term and the approximation term. It can enhance the information flow between different layers, fully preserve the image details, and improve the network convergence speed. Secondly, the integrated attention module was introduced into the reconstruction process as a regularization term. It adaptively fuses the local and non-local features of the image which are used to reconstruct the complex texture and repetitive details of the image, respectively. Note that we innovatively designed a one-shot iteration strategy to simplify the network structure and reduce the reconstruction time while maintaining image quality. Experiments showed that the proposed method is very robust and outperforms state-of-the-art methods in terms of both quantitative and qualitative, greatly improving the preservation of structures and the removal of artifacts.

Exponential Signal Reconstruction With Deep Hankel Matrix Factorization

Exponential function is a basic form of temporal signals, and how to fast acquire this signal is one of the fundamental problems and frontiers in signal processing. To achieve this goal, partial data may be acquired but result in severe artifacts in its spectrum, which is the Fourier transform of exponentials. Thus, reliable spectrum reconstruction is highly expected in the fast data acquisition in many applications, such as chemistry, biology, and medical imaging. In this work, we propose a deep learning method whose neural network structure is designed by imitating the iterative process in the model-based state-of-the-art exponentials' reconstruction method with the low-rank Hankel matrix factorization. With the experiments on synthetic data and realistic biological magnetic resonance signals, we demonstrate that the new method yields much lower reconstruction errors and preserves the low-intensity signals much better than compared methods.

MHAN: Multi-Stage Hybrid Attention Network for MRI reconstruction and super-resolution

High-quality magnetic resonance imaging (MRI) affords clear body tissue structure for reliable diagnosing. However, there is a principal problem of the trade-off between acquisition speed and image quality. Image reconstruction and super-resolution are crucial techniques to solve these problems. In the main field of MR image restoration, most researchers mainly focus on only one of these aspects, namely reconstruction or super-resolution. In this paper, we propose an efficient model called Multi-Stage Hybrid Attention Network (MHAN) that performs the multi-task of recovering high-resolution (HR) MR images from low-resolution (LR) under-sampled measurements. Our model is highlighted by three major modules: (i) an Amplified Spatial Attention Block (ASAB) capable of enhancing the differences in spatial information, (ii) a Self-Attention Block with a Data-Consistency Layer (DC-SAB), which can improve the accuracy of the extracted feature information, (iii) an Adaptive Local Residual Attention Block (ALRAB) that focuses on both spatial and channel information. MHAN employs an encoder-decoder architecture to deeply extract contextual information and a pipeline to provide spatial accuracy. Compared with the recent multi-task model T2Net, our MHAN improves by 2.759 dB in PSNR and 0.026 in SSIM with scaling factor ×2 and acceleration factor 4× on T2 modality.

A Theoretical Framework for Self-Supervised MR Image Reconstruction Using Sub-Sampling via Variable Density Noisier2Noise

In recent years, there has been attention on leveraging the statistical modeling capabilities of neural networks for reconstructing sub-sampled Magnetic Resonance Imaging (MRI) data. Most proposed methods assume the existence of a representative fully-sampled dataset and use fully-supervised training. However, for many applications, fully sampled training data is not available, and may be highly impractical to acquire. The development and understanding of self-supervised methods, which use only sub-sampled data for training, are therefore highly desirable. This work extends the Noisier2Noise framework, which was originally constructed for self-supervised denoising tasks, to variable density sub-sampled MRI data. We use the Noisier2Noise framework to analytically explain the performance of Self-Supervised Learning via Data Undersampling (SSDU), a recently proposed method that performs well in practice but until now lacked theoretical justification. Further, we propose two modifications of SSDU that arise as a consequence of the theoretical developments. Firstly, we propose partitioning the sampling set so that the subsets have the same type of distribution as the original sampling mask. Secondly, we propose a loss weighting that compensates for the sampling and partitioning densities. On the fastMRI dataset we show that these changes significantly improve SSDU's image restoration quality and robustness to the partitioning parameters.

Fast and low-dose medical imaging generation empowered by hybrid deep-learning and iterative reconstruction

Fast and low-dose reconstructions of medical images are highly desired in clinical routines. We propose a hybrid deep-learning and iterative reconstruction (hybrid DL-IR) framework and apply it for fast magnetic resonance imaging (MRI), fast positron emission tomography (PET), and low-dose computed tomography (CT) image generation tasks. First, in a retrospective MRI study (6,066 cases), we demonstrate its capability of handling 3- to 10-fold under-sampled MR data, enabling organ-level coverage with only 10- to 100-s scan time; second, a low-dose CT study (142 cases) shows that our framework can successfully alleviate the noise and streak artifacts in scans performed with only 10% radiation dose (0.61 mGy); and last, a fast whole-body PET study (131 cases) allows us to faithfully reconstruct tumor-induced lesions, including small ones (<4 mm), from 2- to 4-fold-accelerated PET acquisition (30-60 s/bp). This study offers a promising avenue for accurate and high-quality image reconstruction with broad clinical value.

Interferometer response characterization algorithm for multi-aperture Fabry-Perot imaging spectrometers

In recent years, the demand for hyperspectral imaging devices has grown significantly, driven by their ability of capturing high-resolution spectral information. Among the several possible optical designs for acquiring hyperspectral images, there is a growing interest in interferometric spectral imaging systems based on division of aperture. These systems have the advantage of capturing snapshot acquisitions while maintaining a compact design. However, they require a careful calibration to operate properly. In this work, we present the interferometer response characterization algorithm (IRCA), a robust three-step procedure designed to characterize the transmittance response of multi-aperture imaging spectrometers based on the interferometry of Fabry-Perot. Additionally, we propose a formulation of the image formation model for such devices suitable to estimate the parameters of interest by considering the model under various regimes of finesse. The proposed algorithm processes the image output obtained from a set of monochromatic light sources and refines the results using nonlinear regression after an ad-hoc initialization. Through experimental analysis conducted on four different prototypes from the Image SPectrometer On Chip (ImSPOC) family, we validate the performance of our approach for characterization. The associated source code for this paper is available from Zenodo (http://dx.doi.org/10.5281/zenodo.7978514).

Adaptive diffusion priors for accelerated MRI reconstruction

Deep MRI reconstruction is commonly performed with conditional models that de-alias undersampled acquisitions to recover images consistent with fully-sampled data. Since conditional models are trained with knowledge of the imaging operator, they can show poor generalization across variable operators. Unconditional models instead learn generative image priors decoupled from the operator to improve reliability against domain shifts related to the imaging operator. Recent diffusion models are particularly promising given their high sample fidelity. Nevertheless, inference with a static image prior can perform suboptimally. Here we propose the first adaptive diffusion prior for MRI reconstruction, AdaDiff, to improve performance and reliability against domain shifts. AdaDiff leverages an efficient diffusion prior trained via adversarial mapping over large reverse diffusion steps. A two-phase reconstruction is executed following training: a rapid-diffusion phase that produces an initial reconstruction with the trained prior, and an adaptation phase that further refines the result by updating the prior to minimize data-consistency loss. Demonstrations on multi-contrast brain MRI clearly indicate that AdaDiff outperforms competing conditional and unconditional methods under domain shifts, and achieves superior or on par within-domain performance.

Distributed Agent Optimization for Large-Scale Network Models

Optimization of production networks is key for managing efficient hydrocarbon production as part of closed-loop asset management. Large-scale surface network optimization is a challenging task that involves high nonlinearity with numerous constraints. In existing tools, the computational cost of solving the surface network optimization can exponentially increase with the size and complexities of the network using traditional approaches involving nonlinear programming methods. In this study, we accelerate the large-scale surface network optimization by using a distributed agent optimization algorithm called alternating direction method of multipliers (ADMM).

We develop and apply the ADMM algorithm for large-scale network optimization with over 1000 wells and interconnecting pipelines. In the ADMM framework, a large-scale network system is broken down into many small sub-network systems. Then, a smaller optimization problem is formulated for each sub-network. These sub-network optimization problems are solved in parallel using multiple computer cores so that the entire system optimization will be accelerated. A large-scale surface network involves many inequality and equality constraints, which are effectively handled by using augmented Lagrangian method to enhance the robustness of convergence quality. Additionally, proxy or hybrid models can also be used for pipe flow and pressure calculation for every network segment to further speed up the optimization.

The proposed ADMM optimization method is validated by several synthetic cases. We first apply the proposed method to surface network simulation problems of various sizes and complexities (configurations, fluid types, pressure regimes, etc.), where the pressure for all nodes and fluxes in all links will be calculated with a specified separator pressure and reservoir pressures. High accuracy was obtained from the ADMM framework compared with a commercial simulator. Next, the ADMM is applied to network optimization problems, where we optimize the pressure drop across a surface choke for every well to maximize oil production. In a large-scale network case with over 1000 wells, we achieve 2X – 3X speedups in computation time with reasonable accuracy from the ADMM framework compared with benchmarks. Finally, we apply the proposed method to a field case, and validate that the ADMM framework properly works for the actual field applications.

A novel framework for surface network optimization was developed using the distributed agent optimization algorithm. The proposed framework provides superior computational efficiency for large- scale network optimization problems compared with existing benchmark methods. It enables more efficient and frequent decision-making of large-scale petroleum field management to maximize the hydrocarbon production subject to numerous system constraints.

Fast Multi-Contrast MRI Acquisition by Optimal Sampling of Information Complementary to Pre-Acquired MRI Contrast

Recent studies on multi-contrast MRI reconstruction have demonstrated the potential of further accelerating MRI acquisition by exploiting correlation between contrasts. Most of the state-of-the-art approaches have achieved improvement through the development of network architectures for fixed under-sampling patterns, without considering inter-contrast correlation in the under-sampling pattern design. On the other hand, sampling pattern learning methods have shown better reconstruction performance than those with fixed under-sampling patterns. However, most under-sampling pattern learning algorithms are designed for single contrast MRI without exploiting complementary information between contrasts. To this end, we propose a framework to optimize the under-sampling pattern of a target MRI contrast which complements the acquired fully-sampled reference contrast. Specifically, a novel image synthesis network is introduced to extract the redundant information contained in the reference contrast, which is exploited in the subsequent joint pattern optimization and reconstruction network. We have demonstrated superior performance of our learned under-sampling patterns on both public and in-house datasets, compared to the commonly used under-sampling patterns and state-of-the-art methods that jointly optimize the reconstruction network and the under-sampling patterns, up to 8-fold under-sampling factor.

HFIST-Net: High-throughput fast iterative shrinkage thresholding network for accelerating MR image reconstruction

BACKGROUND AND OBJECTIVES

Compressed sensing (CS) is often used to accelerate magnetic resonance image (MRI) reconstruction from undersampled k-space data. A novelty deeply unfolded networks (DUNs) based method, designed by unfolding a traditional CS-MRI optimization algorithm into deep networks, can provide significantly faster reconstruction speeds than traditional CS-MRI methods while improving image quality.

METHODS

In this paper, we propose a High-Throughput Fast Iterative Shrinkage Thresholding Network (HFIST-Net) for reconstructing MR images from sparse measurements by combining traditional model-based CS techniques and data-driven deep learning methods. Specifically, the conventional Fast Iterative Shrinkage Thresholding Algorithm (FISTA) method is expanded as a deep network. To break the bottleneck of information transmission, a multi-channel fusion mechanism is proposed to improve the efficiency of information transmission between adjacent network stages. Moreover, a simple yet efficient channel attention block, called Gaussian context transformer (GCT), is proposed to improve the characterization capabilities of deep Convolutional Neural Network (CNN,) which utilizes Gaussian functions that satisfy preset relationships to achieve context feature excitation.

RESULTS

T1 and T2 brain MR images from the FastMRI dataset are used to validate the performance of the proposed HFIST-Net. The qualitative and quantitative results showed that our method is superior to those compared state-of-the-art unfolded deep learning networks.

CONCLUSIONS

The proposed HFIST-Net is capable of reconstructing more accurate MR image details from highly undersampled k-space data while maintaining fast computational speed.

AutoBCS: Block-Based Image Compressive Sensing With Data-Driven Acquisition and Noniterative Reconstruction

Block compressive sensing (CS) is a well-known signal acquisition and reconstruction paradigm with widespread application prospects in science, engineering, and cybernetic systems. However, state-of-the-art block-based image CS (BCS) methods generally suffer from two issues. The sparsifying domain and the sensing matrices widely used for image acquisition are not data driven and, thus, both the features of the image and the relationships among subblock images are ignored. Moreover, it requires to address a high-dimensional optimization problem with extensive computational complexity for image reconstruction. In this article, we provide a deep learning (DL) strategy for BCS, called AutoBCS, which automatically takes the prior knowledge of images into account in the acquisition step and establishes a reconstruction model for performing fast image reconstruction. More precisely, we present a learning-based sensing matrix to accomplish image acquisition, thereby capturing and preserving more image characteristics than those captured by the existing methods. In addition, we build a noniterative reconstruction network, which provides an end-to-end BCS reconstruction framework to maximize image reconstruction efficiency. Furthermore, we investigate comprehensive comparison studies with both traditional BCS approaches and newly developed DL methods. Compared with these approaches, our proposed AutoBCS can not only provide superior performance in terms of image quality metrics (SSIM and PSNR) and visual perception but also automatically benefit reconstruction speed.

Low-dose sinogram restoration enabled by conditional GAN with cross-domain regularization in SPECT imaging

In order to generate high-quality single-photon emission computed tomography (SPECT) images under low-dose acquisition mode, a sinogram denoising method was studied for suppressing random oscillation and enhancing contrast in the projection domain. A conditional generative adversarial network with cross-domain regularization (CGAN-CDR) is proposed for low-dose SPECT sinogram restoration. The generator stepwise extracts multiscale sinusoidal features from a low-dose sinogram, which are then rebuilt into a restored sinogram. Long skip connections are introduced into the generator, so that the low-level features can be better shared and reused, and the spatial and angular sinogram information can be better recovered. A patch discriminator is employed to capture detailed sinusoidal features within sinogram patches; thereby, detailed features in local receptive fields can be effectively characterized. Meanwhile, a cross-domain regularization is developed in both the projection and image domains. Projection-domain regularization directly constrains the generator via penalizing the difference between generated and label sinograms. Image-domain regularization imposes a similarity constraint on the reconstructed images, which can ameliorate the issue of ill-posedness and serves as an indirect constraint on the generator. By adversarial learning, the CGAN-CDR model can achieve high-quality sinogram restoration. Finally, the preconditioned alternating projection algorithm with total variation regularization is adopted for image reconstruction. Extensive numerical experiments show that the proposed model exhibits good performance in low-dose sinogram restoration. From visual analysis, CGAN-CDR performs well in terms of noise and artifact suppression, contrast enhancement and structure preservation, particularly in low-contrast regions. From quantitative analysis, CGAN-CDR has obtained superior results in both global and local image quality metrics. From robustness analysis, CGAN-CDR can better recover the detailed bone structure of the reconstructed image for a higher-noise sinogram. This work demonstrates the feasibility and effectiveness of CGAN-CDR in low-dose SPECT sinogram restoration. CGAN-CDR can yield significant quality improvement in both projection and image domains, which enables potential applications of the proposed method in real low-dose study.

Connections Between Deep Equilibrium and Sparse Representation Models With Application to Hyperspectral Image Denoising

In this study, the problem of computing a sparse representation of multi-dimensional visual data is considered. In general, such data e.g., hyperspectral images, color images or video data consists of signals that exhibit strong local dependencies. A new computationally efficient sparse coding optimization problem is derived by employing regularization terms that are adapted to the properties of the signals of interest. Exploiting the merits of the learnable regularization techniques, a neural network is employed to act as structure prior and reveal the underlying signal dependencies. To solve the optimization problem Deep unrolling and Deep equilibrium based algorithms are developed, forming highly interpretable and concise deep-learning-based architectures, that process the input dataset in a block-by-block fashion. Extensive simulation results, in the context of hyperspectral image denoising, are provided, which demonstrate that the proposed algorithms outperform significantly other sparse coding approaches and exhibit superior performance against recent state-of-the-art deep-learning-based denoising models. In a wider perspective, our work provides a unique bridge between a classic approach, that is the sparse representation theory, and modern representation tools that are based on deep learning modeling.

Deep Null Space Learning Improves Dataset Recovery for High Frame Rate Synthetic Transmit Aperture Imaging

Synthetic transmit aperture (STA) imaging benefits from the two-way dynamic focusing to achieve optimal lateral resolution and contrast resolution in the full field of view, at the cost of low frame rate (FR) and low signal-to-noise ratio (SNR). In our previous studies, compressed sensing-based STA (CS-STA) and minimal ${l}_{{2}}$ -norm least squares (LS-STA) methods were proposed to recover the complete STA dataset from fewer Hadamard-encoded (HE) plane wave (PW) transmissions. Results demonstrated that, compared with STA imaging, CS/LS-STA can maintain the high resolution of STA in the full field of view and improve the contrast in the deep region with increased FR. However, these methods would introduce errors to the recovered STA datasets and subsequently produce severe artifacts to the beamformed images, especially in the shallow region. Recently, we discovered that the theoretical explanation for the error introduced in the LS-STA-based recovery is that the LS-STA method neglects the null space component of the real STA dataset. To deal with this problem, we propose to train a convolutional neural network under the null space learning framework (CNN-Null) to estimate the missing null space component) for high-accuracy recovery of the STA dataset from fewer HE PW transmissions. The mapping between the low-quality STA dataset (i.e., the range space component of the real STA dataset recovered using the LS-STA method) and the missing null space component of the real STA dataset was learned by the network with the high-quality STA dataset (obtained using full HE STA (HE-STA) imaging) as training labels. The performance of the proposed CNN-Null method was compared with the baseline LS-STA, conventional STA, and HE-STA methods, in terms of the visual quality, the normalized root mean square error (NRMSE), the generalized contrast-to-noise ratio (gCNR), and the lateral full-width at half-maximum (FWHM). The results demonstrate that the proposed method can greatly improve the recovery accuracy of the STA datasets (lower NRMSE) and, therefore, effectively suppress the artifacts presented in the images (especially in the shallow region) obtained using the LS-STA method (with a gCNR improvement of 0.4 in the cross-sectional carotid artery images). In addition, the proposed method can maintain the high lateral resolution of STA with fewer (as low as 16) PW transmissions, as LS-STA does.

A microstructure estimation Transformer inspired by sparse representation for diffusion MRI

Diffusion magnetic resonance imaging (dMRI) is an important tool in characterizing tissue microstructure based on biophysical models, which are typically multi-compartmental models with mathematically complex and highly non-linear forms. Resolving microstructures from these models with conventional optimization techniques is prone to estimation errors and requires dense sampling in the q-space with a long scan time. Deep learning based approaches have been proposed to overcome these limitations. Motivated by the superior performance of the Transformer in feature extraction than the convolutional structure, in this work, we present a learning-based framework based on Transformer, namely, a Microstructure Estimation Transformer with Sparse Coding (METSC) for dMRI-based microstructural parameter estimation. To take advantage of the Transformer while addressing its limitation in large training data requirement, we explicitly introduce an inductive bias-model bias into the Transformer using a sparse coding technique to facilitate the training process. Thus, the METSC is composed with three stages, an embedding stage, a sparse representation stage, and a mapping stage. The embedding stage is a Transformer-based structure that encodes the signal in a high-level space to ensure the core voxel of a patch is represented effectively. In the sparse representation stage, a dictionary is constructed by solving a sparse reconstruction problem that unfolds the Iterative Hard Thresholding (IHT) process. The mapping stage is essentially a decoder that computes the microstructural parameters from the output of the second stage, based on the weighted sum of normalized dictionary coefficients where the weights are also learned. We tested our framework on two dMRI models with downsampled q-space data, including the intravoxel incoherent motion (IVIM) model and the neurite orientation dispersion and density imaging (NODDI) model. The proposed method achieved up to 11.25 folds of acceleration while retaining high fitting accuracy for NODDI fitting, reducing the mean squared error (MSE) up to 70% compared with the previous q-space learning approach. METSC outperformed the other state-of-the-art learning-based methods, including the model-free and model-based methods. The network also showed robustness against noise and generalizability across different datasets. The superior performance of METSC indicates its potential to improve dMRI acquisition and model fitting in clinical applications.

Hybrid ISTA: Unfolding ISTA With Convergence Guarantees Using Free-Form Deep Neural Networks

It is promising to solve linear inverse problems by unfolding iterative algorithms (e.g., iterative shrinkage thresholding algorithm (ISTA)) as deep neural networks (DNNs) with learnable parameters. However, existing ISTA-based unfolded algorithms restrict the network architectures for iterative updates with the partial weight coupling structure to guarantee convergence. In this paper, we propose hybrid ISTA to unfold ISTA with both pre-computed and learned parameters by incorporating free-form DNNs (i.e., DNNs with arbitrary feasible and reasonable network architectures), while ensuring theoretical convergence. We first develop HCISTA to improve the efficiency and flexibility of classical ISTA (with pre-computed parameters) without compromising the convergence rate in theory. Furthermore, the DNN-based hybrid algorithm is generalized to popular variants of learned ISTA, dubbed HLISTA, to enable a free architecture of learned parameters with a guarantee of linear convergence. To our best knowledge, this paper is the first to provide a convergence-provable framework that enables free-form DNNs in ISTA-based unfolded algorithms. This framework is general to endow arbitrary DNNs for solving linear inverse problems with convergence guarantees. Extensive experiments demonstrate that hybrid ISTA can reduce the reconstruction error with an improved convergence rate in the tasks of sparse recovery and compressive sensing.

Deep Gauss-Newton for phase retrieval

We propose the deep Gauss-Newton (DGN) algorithm. The DGN allows one to take into account the knowledge of the forward model in a deep neural network by unrolling a Gauss-Newton optimization method. No regularization or step size needs to be chosen; they are learned through convolutional neural networks. The proposed algorithm does not require an initial reconstruction and is able to retrieve simultaneously the phase and absorption from a single-distance diffraction pattern. The DGN method was applied to both simulated and experimental data and permitted large improvements of the reconstruction error and of the resolution compared with a state-of-the-art iterative method and another neural-network-based reconstruction algorithm.

Unrolled-DOT: an interpretable deep network for diffuse optical tomography

Significance

Imaging through scattering media is critical in many biomedical imaging applications, such as breast tumor detection and functional neuroimaging. Time-of-flight diffuse optical tomography (ToF-DOT) is one of the most promising methods for high-resolution imaging through scattering media. ToF-DOT and many traditional DOT methods require an image reconstruction algorithm. Unfortunately, this algorithm often requires long computational runtimes and may produce lower quality reconstructions in the presence of model mismatch or improper hyperparameter tuning.

Aim

We used a data-driven unrolled network as our ToF-DOT inverse solver. The unrolled network is faster than traditional inverse solvers and achieves higher reconstruction quality by accounting for model mismatch.

Approach

Our model "Unrolled-DOT" uses the learned iterative shrinkage thresholding algorithm. In addition, we incorporate a refinement U-Net and Visual Geometry Group (VGG) perceptual loss to further increase the reconstruction quality. We trained and tested our model on simulated and real-world data and benchmarked against physics-based and learning-based inverse solvers.

Results

In experiments on real-world data, Unrolled-DOT outperformed learning-based algorithms and achieved over 10× reduction in runtime and mean-squared error, compared to traditional physics-based solvers.

Conclusion

We demonstrated a learning-based ToF-DOT inverse solver that achieves state-of-the-art performance in speed and reconstruction quality, which can aid in future applications for noninvasive biomedical imaging.

Compressive Sensing of Medical Images Based on HSV Color Space

Recently, compressive sensing (CS) schemes have been studied as a new compression modality that exploits the sensing matrix in the measurement scheme and the reconstruction scheme to recover the compressed signal. In addition, CS is exploited in medical imaging (MI) to support efficient sampling, compression, transmission, and storage of a large amount of MI. Although CS of MI has been extensively investigated, the effect of color space in CS of MI has not yet been studied in the literature. To fulfill these requirements, this article proposes a novel CS of MI based on hue-saturation value (HSV), using spread spectrum Fourier sampling (SSFS) and sparsity averaging with reweighted analysis (SARA). An HSV loop that performs SSFS is proposed to obtain a compressed signal. Next, HSV-SARA is proposed to reconstruct MI from the compressed signal. A set of color MIs is investigated, such as colonoscopy, magnetic resonance imaging of the brain and eye, and wireless capsule endoscopy images. Experiments were performed to show the superiority of HSV-SARA over benchmark methods in terms of signal-to-noise ratio (SNR), structural similarity (SSIM) index, and measurement rate (MR). The experiments showed that a color MI, with a resolution of 256×256 pixels, could be compressed by the proposed CS at MR of 0.1, and could be improved in terms of SNR being 15.17% and SSIM being 2.53%. The proposed HSV-SARA can be a solution for color medical image compression and sampling to improve the image acquisition of medical devices.

An Efficient Deep Learning-Based High-Definition Image Compressed Sensing Framework for Large-Scene Construction Site Monitoring

High-definition images covering entire large-scene construction sites are increasingly used for monitoring management. However, the transmission of high-definition images is a huge challenge for construction sites with harsh network conditions and scarce computing resources. Thus, an effective compressed sensing and reconstruction method for high-definition monitoring images is urgently needed. Although current deep learning-based image compressed sensing methods exhibit superior performance in recovering images from a reduced number of measurements, they still face difficulties in achieving efficient and accurate high-definition image compressed sensing with less memory usage and computational cost at large-scene construction sites. This paper investigated an efficient deep learning-based high-definition image compressed sensing framework (EHDCS-Net) for large-scene construction site monitoring, which consists of four parts, namely the sampling, initial recovery, deep recovery body, and recovery head subnets. This framework was exquisitely designed by rational organization of the convolutional, downsampling, and pixelshuffle layers based on the procedures of block-based compressed sensing. To effectively reduce memory occupation and computational cost, the framework utilized nonlinear transformations on downscaled feature maps in reconstructing images. Moreover, the efficient channel attention (ECA) module was introduced to further increase the nonlinear reconstruction capability on downscaled feature maps. The framework was tested on large-scene monitoring images from a real hydraulic engineering megaproject. Extensive experiments showed that the proposed EHDCS-Net framework not only used less memory and floating point operations (FLOPs), but it also achieved better reconstruction accuracy with faster recovery speed than other state-of-the-art deep learning-based image compressed sensing methods.

Cascade of Denoising and Mapping Neural Networks for MRI R2* Relaxometry of Iron-Loaded Liver

MRI of effective transverse relaxation rate (R2*) measurement is a reliable method for liver iron concentration quantification. However, R2* mapping can be degraded by noise, especially in the case of iron overload. This study aimed to develop a deep learning method for MRI R2* relaxometry of an iron-loaded liver using a two-stage cascaded neural network. The proposed method, named CadamNet, combines two convolutional neural networks separately designed for image denoising and parameter mapping into a cascade framework, and the physics-based R2* decay model was incorporated in training the mapping network to enforce data consistency further. CadamNet was trained using simulated liver data with Rician noise, which was constructed from clinical liver data. The performance of CadamNet was quantitatively evaluated on simulated data with varying noise levels as well as clinical liver data and compared with the single-stage parameter mapping network (MappingNet) and two conventional model-based R2* mapping methods. CadamNet consistently achieved high-quality R2* maps and outperformed MappingNet at varying noise levels. Compared with conventional R2* mapping methods, CadamNet yielded R2* maps with lower errors, higher quality, and substantially increased efficiency. In conclusion, the proposed CadamNet enables accurate and efficient iron-loaded liver R2* mapping, especially in the presence of severe noise.

SwinGAN: A dual-domain Swin Transformer-based generative adversarial network for MRI reconstruction

Magnetic resonance imaging (MRI) is one of the most important modalities for clinical diagnosis. However, the main disadvantages of MRI are the long scanning time and the moving artifact caused by patient movement during prolonged imaging. It can also lead to patient anxiety and discomfort, so accelerated imaging is indispensable for MRI. Convolutional neural network (CNN) based methods have become the fact standard for medical image reconstruction, and generative adversarial network (GAN) have also been widely used. Nevertheless, due to the limited ability of CNN to capture long-distance information, it may lead to defects in the structure of the reconstructed images such as blurry contour. In this paper, we propose a novel Swin Transformer-based dual-domain generative adversarial network (SwinGAN) for accelerated MRI reconstruction. The SwinGAN consists of two generators: a frequency-domain generator and an image-domain generator. Both the generators utilize Swin Transformer as backbone for effectively capturing the long-distance dependencies. A contextual image relative position encoder (ciRPE) is designed to enhance the ability to capture local information. We extensively evaluate the method on the IXI brain dataset, MICCAI 2013 dataset and MRNet knee dataset. Compared with KIGAN, the peak signal-to-noise ratio (PSNR) and structural similarity index measure (SSIM) are improved by 6.1% and 1.49% to 37.64 dB and 0.98 on IXI dataset respectively, which demonstrates that our model can sufficiently utilize the local and global information of image. The model shows promising performance and robustness under different undersampling masks, different acceleration rates and different datasets. But it needs high hardware requirements with the increasing of the network parameters. The code is available at: https://github.com/learnerzx/SwinGAN.

Parallel non-Cartesian spatial-temporal dictionary learning neural networks (stDLNN) for accelerating 4D-MRI

Dynamic magnetic resonance imaging (MRI) acquisitions are relatively slow due to physical and physiological limitations. The spatial-temporal dictionary learning (DL) approach accelerates dynamic MRI by learning spatial-temporal correlations, but the regularization parameters need to be manually adjusted, the performance at high acceleration rate is limited, and the reconstruction can be time-consuming. Deep learning techniques have shown good performance in accelerating MRI due to the powerful representational capabilities of neural networks. In this work, we propose a parallel non-Cartesian spatial-temporal dictionary learning neural networks (stDLNN) framework that combines dictionary learning with deep learning algorithms and utilizes the spatial-temporal prior information of dynamic MRI data to achieve better reconstruction quality and efficiency. The coefficient estimation modules (CEM) are designed in the framework to adaptively adjust the regularization coefficients. Experimental results show that combining dictionary learning with deep neural networks and using spatial-temporal dictionaries can obviously improve the image quality and computational efficiency compared with the state-of-the-art non-Cartesian imaging methods for accelerating the 4D-MRI especially at high acceleration rate.

Fast iterative reconstruction for photoacoustic tomography using learned physical model: Theoretical validation

Iterative reconstruction has demonstrated superior performance in medical imaging under compressed, sparse, and limited-view sensing scenarios. However, iterative reconstruction algorithms are slow to converge and rely heavily on hand-crafted parameters to achieve good performance. Many iterations are usually required to reconstruct a high-quality image, which is computationally expensive due to repeated evaluations of the physical model. While learned iterative reconstruction approaches such as model-based learning (MBLr) can reduce the number of iterations through convolutional neural networks, it still requires repeated evaluations of the physical models at each iteration. Therefore, the goal of this study is to develop a Fast Iterative Reconstruction (FIRe) algorithm that incorporates a learned physical model into the learned iterative reconstruction scheme to further reduce the reconstruction time while maintaining robust reconstruction performance. We also propose an efficient training scheme for FIRe, which releases the enormous memory footprint required by learned iterative reconstruction methods through the concept of recursive training. The results of our proposed method demonstrate comparable reconstruction performance to learned iterative reconstruction methods with a 9x reduction in computation time and a 620x reduction in computation time compared to variational reconstruction.

A Joint Group Sparsity-based deep learning for multi-contrast MRI reconstruction

Multi-contrast magnetic resonance imaging (MRI) can provide richer diagnosis information. The data acquisition time, however, is increased than single-contrast imaging. To reduce this time, k-space undersampling is an effective way but a smart reconstruction algorithm is required to remove undersampling image artifacts. Traditional algorithms commonly explore the similarity of multi-contrast images through joint sparsity. However, these algorithms are time-consuming due to the iterative process and require adjusting hyperparameters manually. Recently, data-driven deep learning successfully overcome these limitations but the reconstruction error still needs to be further reduced. Here, we propose a Joint Group Sparsity-based Network (JGSN) for multi-contrast MRI reconstruction, which unrolls the iterative process of the joint sparsity algorithm. The designed network includes data consistency modules, learnable sparse transform modules, and joint group sparsity constraint modules. In particular, weights of different contrasts in the transform module are shared to reduce network parameters without sacrificing the quality of reconstruction. The experiments were performed on the retrospective undersampled brain and knee data. Experimental results on in vivo brain data and knee data show that our method consistently outperforms the state-of-the-art methods under different sampling patterns.

One-Dimensional Deep Low-Rank and Sparse Network for Accelerated MRI

Deep learning has shown astonishing performance in accelerated magnetic resonance imaging (MRI). Most state-of-the-art deep learning reconstructions adopt the powerful convolutional neural network and perform 2D convolution since many magnetic resonance images or their corresponding k-space are in 2D. In this work, we present a new approach that explores the 1D convolution, making the deep network much easier to be trained and generalized. We further integrate the 1D convolution into the proposed deep network, named as One-dimensional Deep Low-rank and Sparse network (ODLS), which unrolls the iteration procedure of a low-rank and sparse reconstruction model. Extensive results on in vivo knee and brain datasets demonstrate that, the proposed ODLS is very suitable for the case of limited training subjects and provides improved reconstruction performance than state-of-the-art methods both visually and quantitatively. Additionally, ODLS also shows nice robustness to different undersampling scenarios and some mismatches between the training and test data. In summary, our work demonstrates that the 1D deep learning scheme is memory-efficient and robust in fast MRI.

Noise Characteristics Modeled Unsupervised Network for Robust CT Image Reconstruction

Deep learning (DL)-based methods show great potential in computed tomography (CT) imaging field. The DL-based reconstruction methods are usually evaluated on the training and testing datasets which are obtained from the same distribution, i.e., the same CT scan protocol (i.e., the region setting, kVp, mAs, etc.). In this work, we focus on analyzing the robustness of the DL-based methods against protocol-specific distribution shifts (i.e., the training and testing datasets are from different region settings, different kVp settings, or different mAs settings, respectively). The results show that the DL-based reconstruction methods are sensitive to the protocol-specific perturbations which can be attributed to the noise distribution shift between the training and testing datasets. Based on these findings, we presented a low-dose CT reconstruction method using an unsupervised strategy with the consideration of noise distribution to address the issue of protocol-specific perturbations. Specifically, unpaired sinogram data is enrolled into the network training, which represents unique information for specific imaging protocol, and a Gaussian mixture model (GMM) is introduced to characterize the noise distribution in CT images. It can be termed as GMM based unsupervised CT reconstruction network (GMM-unNet) method. Moreover, an expectation-maximization algorithm is designed to optimize the presented GMM-unNet method. Extensive experiments are performed on three datasets from different scan protocols, which demonstrate that the presented GMM-unNet method outperforms the competing methods both qualitatively and quantitatively.

MR image reconstruction from undersampled data for image-guided radiation therapy using a patient-specific deep manifold image prior

Introduction

Recent advancements in radiotherapy (RT) have allowed for the integration of a Magnetic Resonance (MR) imaging scanner with a medical linear accelerator to use MR images for image guidance to position tumors against the treatment beam. Undersampling in MR acquisition is desired to accelerate the imaging process, but unavoidably deteriorates the reconstructed image quality. In RT, a high-quality MR image of a patient is available for treatment planning. In light of this unique clinical scenario, we proposed to exploit the patient-specific image prior to facilitate high-quality MR image reconstruction.

Methods

Utilizing the planning MR image, we established a deep auto-encoder to form a manifold of image patches of the patient. The trained manifold was then incorporated as a regularization to restore MR images of the same patient from undersampled data. We performed a simulation study using a patient case, a real patient study with three liver cancer patient cases, and a phantom experimental study using data acquired on an in-house small animal MR scanner. We compared the performance of the proposed method with those of the Fourier transform method, a tight-frame based Compressive Sensing method, and a deep learning method with a patient-generic manifold as the image prior.

Results

In the simulation study with 12.5% radial undersampling and 15% increase in noise, our method improved peak-signal-to-noise ratio by 4.46dB and structural similarity index measure by 28% compared to the patient-generic manifold method. In the experimental study, our method outperformed others by producing reconstructions of visually improved image quality.

TransCS: A Transformer-Based Hybrid Architecture for Image Compressed Sensing

Well-known compressed sensing (CS) is widely used in image acquisition and reconstruction. However, accurately reconstructing images from measurements at low sampling rates remains a considerable challenge. In this paper, we propose a novel Transformer-based hybrid architecture (dubbed TransCS) to achieve high-quality image CS. In the sampling module, TransCS adopts a trainable sensing matrix strategy that gains better image reconstruction by learning the structural information from the training images. In the reconstruction module, inspired by the powerful long-distance dependence modelling capacity of the Transformer, a customized iterative shrinkage-thresholding algorithm (ISTA)-based Transformer backbone that iteratively works with gradient descent and soft threshold operation is designed to model the global dependency among image subblocks. Moreover, the auxiliary convolutional neural network (CNN) is introduced to capture the local features of images. Therefore, the proposed hybrid architecture that integrates the customized ISTA-based Transformer backbone with CNN can gain high-performance reconstruction for image compressed sensing. The experimental results demonstrate that our proposed TransCS obtains superior reconstruction quality and noise robustness on several public benchmark datasets compared with other state-of-the-art methods. Our code is available on TransCS.

Iterative deep neural networks based on proximal gradient descent for image restoration

The algorithm unfolding networks with explainability of algorithms and higher efficiency of Deep Neural Networks (DNN) have received considerable attention in solving ill-posed inverse problems. Under the algorithm unfolding network framework, we propose a novel end-to-end iterative deep neural network and its fast network for image restoration. The first one is designed making use of proximal gradient descent algorithm of variational models, which consists of denoiser and reconstruction sub-networks. The second one is its accelerated version with momentum factors. For sub-network of denoiser, we embed the Convolutional Block Attention Module (CBAM) in previous U-Net for adaptive feature refinement. Experiments on image denoising and deblurring demonstrate that competitive performances in quality and efficiency are gained by compared with several state-of-the-art networks for image restoration. Proposed unfolding DNN can be easily extended to solve other similar image restoration tasks, such as image super-resolution, image demosaicking, etc.