Learned Primal-Dual Reconstruction

Steerable Conditional Diffusion for Out-of-Distribution Adaptation in Medical Image Reconstruction

Denoising diffusion models have emerged as the go-to generative framework for solving inverse problems in imaging. A critical concern regarding these models is their performance on out-of-distribution tasks, which remains an under-explored challenge. Using a diffusion model on an out-of-distribution dataset, realistic reconstructions can be generated, but with hallucinating image features that are uniquely present in the training dataset. To address this discrepancy and improve reconstruction accuracy, we introduce a novel test-time adaptation sampling framework called Steerable Conditional Diffusion. Specifically, this framework adapts the diffusion model, concurrently with image reconstruction, based solely on the information provided by the available measurement. Utilising the proposed method, we achieve substantial enhancements in out-of-distribution performance across diverse imaging modalities, advancing the robust deployment of denoising diffusion models in real-world applications.

PEARL: Cascaded Self-Supervised Cross-Fusion Learning for Parallel MRI Acceleration

Supervised deep learning (SDL) methodology holds promise for accelerated magnetic resonance imaging (AMRI) but is hampered by the reliance on extensive training data. Some self-supervised frameworks, such as deep image prior (DIP), have emerged, eliminating the explicit training procedure but often struggling to remove noise and artifacts under significant degradation. This work introduces a novel self-supervised accelerated parallel MRI approach called PEARL, leveraging a multiple-stream joint deep decoder with two cross-fusion schemes to accurately reconstruct one or more target images from compressively sampled k-space. Each stream comprises cascaded cross-fusion sub-block networks (SBNs) that sequentially perform combined upsampling, 2D convolution, joint attention, ReLU activation and batch normalization (BN). Among them, combined upsampling and joint attention facilitate mutual learning between multiple-stream networks by integrating multi-parameter priors in both additive and multiplicative manners. Long-range unified skip connections within SBNs ensure effective information propagation between distant cross-fusion layers. Additionally, incorporating dual-normalized edge-orientation similarity regularization into the training loss enhances detail reconstruction and prevents overfitting. Experimental results consistently demonstrate that PEARL outperforms the existing state-of-the-art (SOTA) self-supervised AMRI technologies in various MRI cases. Notably, 5-fold$\sim$6-fold accelerated acquisition yields a 1$\%$ $\sim$ 2$\%$ improvement in SSIM$_{\mathsf{ROI}}$ and a 3$\%$ $\sim$ 6$\%$ improvement in PSNR$_{\mathsf{ROI}}$, along with a significant 15$\%$ $\sim$ 20$\%$ reduction in RLNE$_{\mathsf{ROI}}$.

Low dose computed tomography reconstruction with momentum-based frequency adjustment network

BACKGROUND AND OBJECTIVE

Recent investigations into Low-Dose Computed Tomography (LDCT) reconstruction methods have brought Model-Based Data-Driven (MBDD) approaches to the forefront. One prominent architecture within MBDD entails the integration of Model-Based Iterative Reconstruction (MBIR) with Deep Learning (DL). While this approach offers the advantage of harnessing information from sinogram and image domains, it also reveals several deficiencies. First and foremost, the efficacy of DL methods within the realm of MBDD necessitates meticulous enhancement, as it directly impacts the computational cost and the quality of reconstructed images. Next, high computational costs and a high number of iterations limit the development of MBDD methods. Last but not least, CT reconstruction is sensitive to pixel accuracy, and the role of loss functions within DL methods is crucial for meeting this requirement.

METHODS

This paper advances MBDD methods through three principal contributions. Firstly, we introduce an innovative Frequency Adjustment Network (FAN) that effectively adjusts both high and low-frequency components during the inference phase, resulting in substantial enhancements in reconstruction performance. Second, we develop the Momentum-based Frequency Adjustment Network (MFAN), which leverages momentum terms as an extrapolation strategy to facilitate the amplification of changes throughout successive iterations, culminating in a rapid convergence framework. Lastly, we delve into the visual properties of CT images and present a unique loss function named Focal Detail Loss (FDL). The FDL function preserves fine details throughout the training phase, significantly improving reconstruction quality.

RESULTS

Through a series of experiments validation on the AAPM-Mayo public dataset and real-world piglet datasets, the aforementioned three contributions demonstrated superior performance. MFAN achieved convergence in 10 iterations as an iteration method, faster than other methods. Ablation studies further highlight the advanced performance of each contribution.

CONCLUSIONS

This paper presents an MBDD-based LDCT reconstruction method using a momentum-based frequency adjustment network with a focal detail loss function. This approach significantly reduces the number of iterations required for convergence while achieving superior reconstruction results in visual and numerical analyses.

Deep Radon Prior: A fully unsupervised framework for sparse-view CT reconstruction

BACKGROUND

Sparse-view computed tomography (CT) substantially reduces radiation exposure but often introduces severe artifacts that compromise image fidelity. Recent advances in deep learning for solving inverse problems have shown considerable promise in enhancing CT reconstruction; however, most approaches heavily rely on high-quality training datasets and lack interpretability.

METHOD

To address these challenges, this paper introduces a novel, fully unsupervised deep learning framework that mitigates the dependency on extensive labeled data and improves the interpretability of the reconstruction process. Specifically, we propose the Deep Radon Prior (DRP) framework, inspired by the Deep Image Prior (DIP), which integrates a neural network as an implicit prior into the iterative reconstruction process. This integration facilitates the image domain and the Radon domain gradient feedback and progressively optimizes the neural network through multiple stages, effectively narrowing the solution space in the Radon domain for under-constrained imaging protocols.

RESULTS

We discuss the convergence properties of DRP and validate our approach experimentally, demonstrating its ability to produce high-fidelity images while significantly reducing artifacts. Results indicate that DRP achieves comparable or superior performance to supervised methods, thereby addressing the inherent challenges of sparse-view CT and substantially enhancing image quality.

CONCLUSIONS

The introduction of DRP represents a significant advancement in sparse-view CT imaging by leveraging the inherent deep self-correlation of the Radon domain, enabling effective cooperation with neural network manifolds for image reconstruction. This paradigm shift toward fully unsupervised learning offers a scalable and insightful approach to medical imaging, potentially redefining the landscape of CT reconstruction.

Fourier-enhanced high-order total variation (FeHOT) iterative network for interior tomography

Objective. Determining a satisfactory solution for different computed tomography (CT) fields has been a long-standing challenge in the interior tomography, Traditional methods like FBP suffer from low contrast, while deep learning approaches often lack data consistency. The goal is to leverage high-order total variation (HOT) regularization and Fourier-based frequency domain enhancement to achieve high-precision reconstruction from truncated projection data while overcoming limitations such as slow convergence, over-smoothing, and loss of high-frequency details in existing methods.Approach. The proposed Fourier-enhanced HOT (FeHOT) network employs a coarse-to-fine strategy. First, a HOT-based unrolled iterative network accelerates coarse reconstruction using a learned primal-dual algorithm for data consistency and implicit high-order gradient constraints. Second, a Fourier-enhanced U-Net module selectively attenuates low-frequency components in skip connections while amplifying high-frequency features from filtered back-projection (FBP) results, preserving edge and texture details. Frequency-dependent scaling factors are introduced to balance spectral components during refinement.Main Results. Experiments on the AAPM and clinical medical datasets demonstrate FeHOT's superiority over competing methods (FBP, HOT, AG-Net, PD-Net). For the medical dataset, FeHOT achieved PSNR = 41.17 (noise-free) and 39.24 (noisy), outperforming PD-Net (33.42/31.08) and AG-Net (33.41/31.31). Meanwhile, For the AAPM dataset, where imaged objects exhibit piecewise constant properties, first-order total variation achieved satisfactory results. In contrast, for clinical medical datasets with non-piecewise-constant characteristics (e.g. complex anatomical structures), FeHOT's second-order regularization better aligned with the high-quality requirements of interior tomography. Ablation studies confirmed the necessity of Fourier enhancement, showing significant improvements in edge preservation (e.g. SSIM increased from 0.9877 to 0.9976 for noise-free cases). The method achieved high-quality reconstruction within five iterations, reducing computational costs.Significance. FeHOT represents a paradigm shift in interior tomography by: 1) Bridging classical HOT theory with deep learning through an iterative unrolling framework. 2) Introducing frequency-domain operations to overcome the limitations of polynomial/piecewise-constant assumptions in CT images. 3) Enabling high-quality reconstruction in just five iterations, balancing computational efficiency with accuracy. This method offers a promising solution for low-dose, precise imaging in clinical and industrial applications.

Accelerating iterative ptychography with an integrated neural network

Electron ptychography is a powerful and versatile tool for high-resolution and dose-efficient imaging. Iterative reconstruction algorithms are powerful but also computationally expensive due to their relative complexity and the many hyperparameters that must be optimised. Gradient descent-based iterative ptychography is a popular method, but it may converge slowly when reconstructing low spatial frequencies. In this work, we present a method for accelerating a gradient descent-based iterative reconstruction algorithm by training a neural network (NN) that is applied in the reconstruction loop. The NN works in Fourier space and selectively boosts low spatial frequencies, thus enabling faster convergence in a manner similar to accelerated gradient descent algorithms. We discuss the difficulties that arise when incorporating a NN into an iterative reconstruction algorithm and show how they can be overcome with iterative training. We apply our method to simulated and experimental data of gold nanoparticles on amorphous carbon and show that we can significantly speed up ptychographic reconstruction of the nanoparticles.

Complex-valued neural networks to speed-up MR thermometry during hyperthermia using Fourier PD and PDUNet

Hyperthermia (HT) in combination with radio- and/or chemotherapy has become an accepted cancer treatment for distinct solid tumour entities. In HT, tumour tissue is exogenously heated to temperatures between 39 and 43 °C for 60 min. Temperature monitoring can be performed non-invasively using dynamic magnetic resonance imaging (MRI). However, the slow nature of MRI leads to motion artefacts in the images due to the movements of patients during image acquisition. By discarding parts of the data, the speed of the acquisition can be increased - known as undersampling. However, due to the invalidation of the Nyquist criterion, the acquired images might be blurry and can also produce aliasing artefacts. The aim of this work was, therefore, to reconstruct highly undersampled MR thermometry acquisitions with better resolution and with fewer artefacts compared to conventional methods. The use of deep learning in the medical field has emerged in recent times, and various studies have shown that deep learning has the potential to solve inverse problems such as MR image reconstruction. However, most of the published work only focuses on the magnitude images, while the phase images are ignored, which are fundamental requirements for MR thermometry. This work, for the first time, presents deep learning-based solutions for reconstructing undersampled MR thermometry data. Two different deep learning models have been employed here, the Fourier Primal-Dual network and the Fourier Primal-Dual UNet, to reconstruct highly undersampled complex images of MR thermometry. MR images of 44 patients with different sarcoma types who received HT treatment in combination with radiotherapy and/or chemotherapy were used in this study. The method reduced the temperature difference between the undersampled MRIs and the fully sampled MRIs from 1.3 to 0.6 °C in full volume and 0.49 °C to 0.06 °C in the tumour region for a theoretical acceleration factor of 10.

Spatial-frequency aware zero-centric residual unfolding network for MRI reconstruction

Magnetic Resonance Imaging is a cornerstone of medical diagnostics, providing high-quality soft tissue contrast through non-invasive methods. However, MRI technology faces critical limitations in imaging speed and resolution. Prolonged scan times not only increase patient discomfort but also contribute to motion artifacts, further compromising image quality. Compressed Sensing (CS) theory has enabled the acquisition of partial k-space data, which can then be effectively reconstructed to recover the original image using advanced reconstruction algorithms. Recently, deep learning has been widely applied to MRI reconstruction, aiming to reduce the artifacts in the image domain caused by undersampling in k-space and enhance image quality. As deep learning continues to evolve, the undersampling factors in k-space have gradually increased in recent years. However, these layers are limited in compensating for reconstruction errors in the unsampled areas, impeding further performance improvements. To address this, we propose a learnable spatial-frequency difference-aware module that complements the learnable data consistency layer, mapping k-space domain differences to the spatial image domain for perceptual compensation. Additionally, inspired by wavelet decomposition, we introduce explicit priors by decomposing images into mean and residual components, enforcing a refined zero-mean constraint on the residuals while maintaining computational efficiency. Comparative experiments on the FastMRI and Calgary-Campinas datasets demonstrate that our method achieves superior reconstruction performance against seven state-of-the-art techniques. Ablation studies further confirm the efficacy of our model's architecture, establishing a new pathway for enhanced MRI reconstruction.

A Study of Why We Need to Reassess Full Reference Image Quality Assessment with Medical Images

Image quality assessment (IQA) is indispensable in clinical practice to ensure high standards, as well as in the development stage of machine learning algorithms that operate on medical images. The popular full reference (FR) IQA measures PSNR and SSIM are known and tested for working successfully in many natural imaging tasks, but discrepancies in medical scenarios have been reported in the literature, highlighting the gap between development and actual clinical application. Such inconsistencies are not surprising, as medical images have very different properties than natural images, and PSNR and SSIM have neither been targeted nor properly tested for medical images. This may cause unforeseen problems in clinical applications due to wrong judgement of novel methods. This paper provides a structured and comprehensive overview of examples where PSNR and SSIM prove to be unsuitable for the assessment of novel algorithms using different kinds of medical images, including real-world MRI, CT, OCT, X-Ray, digital pathology and photoacoustic imaging data. Therefore, improvement is urgently needed in particular in this era of AI to increase reliability and explainability in machine learning for medical imaging and beyond. Lastly, we will provide ideas for future research as well as suggest guidelines for the usage of FR-IQA measures applied to medical images.

Deep Guess acceleration for explainable image reconstruction in sparse-view CT

Sparse-view Computed Tomography (CT) is an emerging protocol designed to reduce X-ray dose radiation in medical imaging. Reconstructions based on the traditional Filtered Back Projection algorithm suffer from severe artifacts due to sparse data. In contrast, Model-Based Iterative Reconstruction (MBIR) algorithms, though better at mitigating noise through regularization, are too computationally costly for clinical use. This paper introduces a novel technique, denoted as the Deep Guess acceleration scheme, using a trained neural network both to quicken the regularized MBIR and to enhance the reconstruction accuracy. We integrate state-of-the-art deep learning tools to initialize a clever starting guess for a proximal algorithm solving a non-convex model and thus computing a (mathematically) interpretable solution image in a few iterations. Experimental results on real and synthetic CT images demonstrate the Deep Guess effectiveness in (very) sparse tomographic protocols, where it overcomes its mere variational counterpart and many data-driven approaches at the state of the art. We also consider a ground truth-free implementation and test the robustness of the proposed framework to noise.

CAMP-Net: Consistency-Aware Multi-Prior Network for Accelerated MRI Reconstruction

Undersampling -space data in magnetic resonance imaging (MRI) reduces scan time but pose challenges in image reconstruction. Considerable progress has been made in reconstructing accelerated MRI. However, restoration of high-frequency image details in highly undersampled data remains challenging. To address this issue, we propose CAMP-Net, an unrolling-based Consistency-Aware Multi-Prior Network for accelerated MRI reconstruction. CAMP-Net leverages complementary multi-prior knowledge and multi-slice information from various domains to enhance reconstruction quality. Specifically, CAMP-Net comprises three interleaved modules for image enhancement, -space restoration, and calibration consistency, respectively. These modules jointly learn priors from data in image domain, -domain, and calibration region, respectively, in data-driven manner during each unrolled iteration. Notably, the encoded calibration prior knowledge extracted from auto-calibrating signals implicitly guides the learning of consistency-aware -space correlation for reliable interpolation of missing -space data. To maximize the benefits of image domain and -domain prior knowledge, the reconstructions are aggregated in a frequency fusion module, exploiting their complementary properties to optimize the trade-off between artifact removal and fine detail preservation. Additionally, we incorporate a surface data fidelity layer during the learning of -domain and calibration domain priors to prevent degradation of the reconstruction caused by padding-induced data imperfections. We evaluate the generalizability and robustness of our method on three large public datasets with varying acceleration factors and sampling patterns. The experimental results demonstrate that our method outperforms state-of-the-art approaches in terms of both reconstruction quality and mapping estimation, particularly in scenarios with high acceleration factors.

Artificial Intelligence in Computed Tomography Image Reconstruction: A Review of Recent Advances

The development of novel image reconstruction algorithms has been pivotal in enhancing image quality and reducing radiation dose in computed tomography (CT) imaging. Traditional techniques like filtered back projection perform well under ideal conditions but fail to generate high-quality images under low-dose, sparse-view, and limited-angle conditions. Iterative reconstruction methods improve upon filtered back projection by incorporating system models and assumptions about the patient, yet they can suffer from patchy image textures. The emergence of artificial intelligence (AI), particularly deep learning, has further advanced CT reconstruction. AI techniques have demonstrated great potential in reducing radiation dose while preserving image quality and noise texture. Moreover, AI has exhibited unprecedented performance in addressing challenging CT reconstruction problems, including low-dose CT, sparse-view CT, limited-angle CT, and interior tomography. This review focuses on the latest advances in AI-based CT reconstruction under these challenging conditions.

Trustworthy Limited Data CT Reconstruction Using Progressive Artifact Image Learning

The reconstruction of limited data computed tomography (CT) aims to obtain high-quality images from a reduced set of projection views acquired from sparse views or limited angles. This approach is utilized to reduce radiation exposure or expedite the scanning process. Deep Learning (DL) techniques have been incorporated into limited data CT reconstruction tasks and achieve remarkable performance. However, these DL methods suffer from various limitations. Firstly, the distribution inconsistency between the simulation data and the real data hinders the generalization of these DL-based methods. Secondly, these DL-based methods could be unstable due to lack of kernel awareness. This paper addresses these issues by proposing an unrolling framework called Progressive Artifact Image Learning (PAIL) for limited data CT reconstruction. The proposed PAIL primarily consists of three key modules, i.e., a residual domain module (RDM), an image domain module (IDM), and a wavelet domain module (WDM). The RDM is designed to refine features from residual images and suppress the observable artifacts from the reconstructed images. This module could effectively alleviate the effects of distribution inconsistency among different data sets by transferring the optimization space from the original data domain to the residual data domain. The IDM is designed to suppress the unobservable artifacts in the image space. The RDM and IDM collaborate with each other during the iterative optimization process, progressively removing artifacts and reconstructing the underlying CT image. Furthermore, in order to void the potential hallucinations generated by the RDM and IDM, an additional WDM is incorporated into the network to enhance its stability. This is achieved by making the network become kernel-aware via integrating wavelet-based compressed sensing. The effectiveness of the proposed PAIL method has been consistently verified on two simulated CT data sets, a clinical cardiac data set and a sheep lung data set. Compared to other state-of-the-art methods, the proposed PAIL method achieves superior performance in various limited data CT reconstruction tasks, demonstrating its promising generalization and stability.

An efficient deep unrolling network for sparse-view CT reconstruction via alternating optimization of dense-view sinograms and images

Objective. Recently, there have been many advancements in deep unrolling methods for sparse-view computed tomography (SVCT) reconstruction. These methods combine model-based and deep learning-based reconstruction techniques, improving the interpretability and achieving significant results. However, they are often computationally expensive, particularly for clinical raw projection data with large sizes. This study aims to address this issue while maintaining the quality of the reconstructed image.Approach. The SVCT reconstruction task is decomposed into two subproblems using the proximal gradient method: optimizing dense-view sinograms and optimizing images. Then dense-view sinogram inpainting, image-residual learning, and image-refinement modules are performed at each iteration stage using deep neural networks. Unlike previous unrolling methods, the proposed method focuses on optimizing dense-view sinograms instead of full-view sinograms. This approach not only reduces computational resources and runtime but also minimizes the challenge for the network to perform sinogram inpainting when the sparse ratio is extremely small, thereby decreasing the propagation of estimation error from the sinogram domain to the image domain.Main results. The proposed method successfully reconstructs an image (512 × 512 pixels) from real-size (2304 × 736) projection data, with 3.39 M training parameters and an inference time of 0.09 s per slice on a GPU. The proposed method also achieves superior quantitative and qualitative results compared with state-of-the-art deep unrolling methods on datasets with sparse ratios of 1/12 and 1/18, especially in suppressing artifacts and preserving structural details. Additionally, results show that using dense-view sinogram inpainting not only accelerates the computational speed but also leads to faster network convergence and further improvements in reconstruction results.Significance. This research presents an efficient dual-domain deep unrolling technique that produces excellent results in SVCT reconstruction while requiring small computational resources. These findings have important implications for speeding up deep unrolling CT reconstruction methods and making them more practical for processing clinical CT projection data.

Maternal ECG-Guided Neural Network for Improved Fetal Electrocardiogram Extraction

Historically, acquiring a reliable and accurate non-invasive fetal electrocardiogram has several significant challenges in both data acquisition and attenuation of maternal signals. These barriers include maternal physical/physiological parameters, hardware sensitivity, and the effectiveness of signal processing algorithms in separating maternal and fetal electrocardiograms. In this paper, we focus on the evaluation of signal-processing algorithms. Here, we propose a learning-based method based on the integration of maternal electrocardiogram acquired as guidance for transabdominal fetal electrocardiogram signal extraction. The results demonstrate that incorporating the maternal electrocardiogram signal as input for training the neural network outperforms the network solely trained using information from the abdominal electrocardiogram. This indicates that leveraging the maternal electrocardiogram serves as a suitable prior for effectively attenuating maternal electrocardiogram from the abdominal electrocardiogram.

Performance evaluation of a 4D similarity filter for dynamic CT angiography imaging of the liver

BACKGROUND

Dynamic computed tomography (CT) angiography of the abdomen provides perfusion information and characteristics of the tissues present in the abdomen. This information could potentially help characterize liver metastases. However, radiation dose has to be relatively low for the patient, causing the images to have very high noise content. Denoising methods are needed to increase image quality.

PURPOSE

The purpose of this study was to investigate the performance, limitations, and behavior of a new 4D filtering method, called the 4D Similarity Filter (4DSF), to reduce image noise in temporal CT data.

METHODS

The 4DSF averages voxels with similar time-intensity curves (TICs). Each phase is filtered individually using the information of all phases except for the one being filtered. This approach minimizes the bias toward the noise initially present in this phase. Since the 4DSF does not base similarity on spatial proximity, loss of spatial resolution is avoided. The 4DSF was evaluated on a 12-phase liver dynamic CT angiography acquisition of 52 digital anthropomorphic phantoms, each containing one hypervascular 1 cm lesion with a small necrotic core. The metrics used for evaluation were noise reduction, lesion contrast-to-noise ratio (CNR), CT number accuracy using peak-time and peak-intensity of the TICs, and resolution loss. The results were compared to those obtained by the time-intensity profile similarity (TIPS) filter, which uses the whole TIC for determining similarity, and the combination 4DSF followed by TIPS filter (4DSF + TIPS).

RESULTS

The 4DSF alone resulted in a median noise reduction by a factor of 6.8, which is lower than that obtained by the TIPS filter at 8.1, and 4DSF + TIPS at 12.2. The 4DSF increased the median CNR from 0. 44 to 1.85, which is less than the TIPS filter at 2.59 and 4DSF + TIPS at 3.12. However, the peak-intensity accuracy in the TICs was superior for the 4DSF, with a median intensity decrease of -34 HU compared to -88 and -50 HU for the hepatic artery when using the TIPS filter and 4DSF + TIPS, respectively. The median peak-time accuracy was inferior for the 4DSF filter and 4DSF + TIPS, with a time shift of -1 phases for the portal vein TIC compared to no shift in time when using the TIPS. The analysis of the full-width-at-half-maximum (FWHM) of a small artery showed significantly less spatial resolution loss for the 4DSF at 3.2 pixels, compared to the TIPS filter at 4.3 pixels, and 3.4 pixels for the 4DSF + TIPS.

CONCLUSION

The 4DSF can reduce noise with almost no resolution loss, making the filter very suitable for denoising 4D CT data for detection tasks, even in very low dose, i.e., very high noise level, situations. In combination with the TIPS filter, the noise reduction can be increased even further.

Deep-blur: Blind identification and deblurring with convolutional neural networks

We propose a neural network architecture and a training procedure to estimate blurring operators and deblur images from a single degraded image. Our key assumption is that the forward operators can be parameterized by a low-dimensional vector. The models we consider include a description of the point spread function with Zernike polynomials in the pupil plane or product-convolution expansions, which incorporate space-varying operators. Numerical experiments show that the proposed method can accurately and robustly recover the blur parameters even for large noise levels. For a convolution model, the average signal-to-noise ratio of the recovered point spread function ranges from 13 dB in the noiseless regime to 8 dB in the high-noise regime. In comparison, the tested alternatives yield negative values. This operator estimate can then be used as an input for an unrolled neural network to deblur the image. Quantitative experiments on synthetic data demonstrate that this method outperforms other commonly used methods both perceptually and in terms of SSIM. The algorithm can process a 512 512 image under a second on a consumer graphics card and does not require any human interaction once the operator parameterization has been set up.1.

An unrolled neural network for accelerated dynamic MRI based on second-order half-quadratic splitting model

The reconstruction of dynamic magnetic resonance images from incomplete k-space data has sparked significant research interest due to its potential to reduce scan time. However, traditional iterative optimization algorithms fail to faithfully reconstruct images at higher acceleration factors and incur long reconstruction time. Furthermore, end-to-end deep learning-based reconstruction algorithms suffer from large model parameters and lack robustness in the reconstruction results. Recently, unrolled deep learning models, have shown immense potential in algorithm stability and applicability flexibility. In this paper, we propose an unrolled deep learning network based on a second-order Half-Quadratic Splitting(HQS) algorithm, where the forward propagation process of this framework strictly follows the computational flow of the HQS algorithm. In particular, we propose a degradation-sense module by associating random sampling patterns with intermediate variables to guide the iterative process. We introduce the Information Fusion Transformer(IFT) to extract both local and non-local prior information from image sequences, thereby removing aliasing artifacts resulting from random undersampling. Finally, we impose low-rank constraints within the HQS algorithm to further enhance the reconstruction results. The experiments demonstrate that each component module of our proposed model contributes to the improvement of the reconstruction task. Our proposed method achieves comparably satisfying performance to the state-of-the-art methods and it exhibits excellent generalization capabilities across different sampling masks. At the low acceleration factor, there is a 0.7% enhancement in the PSNR. Furthermore, when the acceleration factor reached 8 and 12, the PSNR achieves an improvement of 3.4% and 5.8% respectively.

Self-supervised learning for CT image denoising and reconstruction: a review

This article reviews the self-supervised learning methods for CT image denoising and reconstruction. Currently, deep learning has become a dominant tool in medical imaging as well as computer vision. In particular, self-supervised learning approaches have attracted great attention as a technique for learning CT images without clean/noisy references. After briefly reviewing the fundamentals of CT image denoising and reconstruction, we examine the progress of deep learning in CT image denoising and reconstruction. Finally, we focus on the theoretical and methodological evolution of self-supervised learning for image denoising and reconstruction.

ATOMMIC: An Advanced Toolbox for Multitask Medical Imaging Consistency to facilitate Artificial Intelligence applications from acquisition to analysis in Magnetic Resonance Imaging

BACKGROUND AND OBJECTIVES

Artificial intelligence (AI) is revolutionizing Magnetic Resonance Imaging (MRI) along the acquisition and processing chain. Advanced AI frameworks have been applied in various successive tasks, such as image reconstruction, quantitative parameter map estimation, and image segmentation. However, existing frameworks are often designed to perform tasks independently of each other or are focused on specific models or single datasets, limiting generalization. This work introduces the Advanced Toolbox for Multitask Medical Imaging Consistency (ATOMMIC), a novel open-source toolbox that streamlines AI applications for accelerated MRI reconstruction and analysis. ATOMMIC implements several tasks using deep learning (DL) models and enables MultiTask Learning (MTL) to perform related tasks in an integrated manner, targeting generalization in the MRI domain.

METHODS

We conducted a comprehensive literature review and analyzed 12,479 GitHub repositories to assess the current landscape of AI frameworks for MRI. Subsequently, we demonstrate how ATOMMIC standardizes workflows and improves data interoperability, enabling effective benchmarking of various DL models across MRI tasks and datasets. To showcase ATOMMIC's capabilities, we evaluated twenty-five DL models on eight publicly available datasets, focusing on accelerated MRI reconstruction, segmentation, quantitative parameter map estimation, and joint accelerated MRI reconstruction and segmentation using MTL.

RESULTS

ATOMMIC's high-performance training and testing capabilities, utilizing multiple GPUs and mixed precision support, enable efficient benchmarking of multiple models across various tasks. The framework's modular architecture implements each task through a collection of data loaders, models, loss functions, evaluation metrics, and pre-processing transformations, facilitating seamless integration of new tasks, datasets, and models. Our findings demonstrate that ATOMMIC supports MTL for multiple MRI tasks with harmonized complex-valued and real-valued data support while maintaining active development and documentation. Task-specific evaluations demonstrate that physics-based models outperform other approaches in reconstructing highly accelerated acquisitions. These high-quality reconstruction models also show superior accuracy in estimating quantitative parameter maps. Furthermore, when combining high-performing reconstruction models with robust segmentation networks through MTL, performance is improved in both tasks.

CONCLUSIONS

ATOMMIC advances MRI reconstruction and analysis by leveraging MTL and ensuring consistency across tasks, models, and datasets. This comprehensive framework serves as a versatile platform for researchers to use existing AI methods and develop new approaches in medical imaging.

Hybrid Reconstruction Approach for Polychromatic Computed Tomography in Highly Limited-Data Scenarios

Conventional strategies aimed at mitigating beam-hardening artifacts in computed tomography (CT) can be categorized into two main approaches: (1) postprocessing following conventional reconstruction and (2) iterative reconstruction incorporating a beam-hardening model. While the former fails in low-dose and/or limited-data cases, the latter substantially increases computational cost. Although deep learning-based methods have been proposed for several cases of limited-data CT, few works in the literature have dealt with beam-hardening artifacts, and none have addressed the problems caused by randomly selected projections and a highly limited span. We propose the deep learning-based prior image constrained (PICDL) framework, a hybrid method used to yield CT images free from beam-hardening artifacts in different limited-data scenarios based on the combination of a modified version of the Prior Image Constrained Compressed Sensing (PICCS) algorithm that incorporates the L2 norm (L2-PICCS) with a prior image generated from a preliminary FDK reconstruction with a deep learning (DL) algorithm. The model is based on a modification of the U-Net architecture, incorporating ResNet-34 as a replacement of the original encoder. Evaluation with rodent head studies in a small-animal CT scanner showed that the proposed method was able to correct beam-hardening artifacts, recover patient contours, and compensate streak and deformation artifacts in scenarios with a limited span and a limited number of projections randomly selected. Hallucinations present in the prior image caused by the deep learning model were eliminated, while the target information was effectively recovered by the L2-PICCS algorithm.

Prior-image-based low-dose CT reconstruction for adaptive radiation therapy

Objective. The study aims to reduce the imaging radiation dose in Adaptive Radiotherapy (ART) while maintaining high-quality CT images, critical for effective treatment planning and monitoring.Approach. We developed the Prior-aware Learned Primal-Dual Network (pLPD-UNet), which uses prior CT images to enhance reconstructions from low-dose scans. The network was separately trained on thorax and abdomen datasets to accommodate the unique imaging requirements of each anatomical region.Main results. The pLPD-UNet demonstrated improved reconstruction accuracy and robustness in handling sparse data compared to traditional methods. It effectively maintained image quality essential for precise organ delineation and dose calculation, while achieving a significant reduction in radiation exposure.Significance. This method offers a significant advancement in the practice of ART by integrating prior imaging data, potentially setting a new standard for balancing radiation safety with the need for high-resolution imaging in cancer treatment planning.

Designing Universally-Approximating Deep Neural Networks: A First-Order Optimization Approach

Universal approximation capability, also referred to as universality, is an important property of deep neural networks, endowing them with the potency to accurately represent the underlying target function in learning tasks. In practice, the architecture of deep neural networks largely influences the performance of the models. However, most existing methodologies for designing neural architectures, such as the heuristic manual design or neural architecture search, ignore the universal approximation property, thus losing a potential safeguard about the performance. In this paper, we propose a unified framework to design the architectures of deep neural networks with a universality guarantee based on first-order optimization algorithms, where the forward pass is interpreted as the updates of an optimization algorithm. The (explicit or implicit) network is designed by replacing each gradient term in the algorithm with a learnable module similar to a two-layer network or its derivatives. Specifically, we explore the realm of width-bounded neural networks, a common practical scenario, showcasing their universality. Moreover, adding operations of normalization, downsampling, and upsampling does not hurt the universality. To the best of our knowledge, this is the first work that width-bounded networks with universal approximation guarantee can be designed in a principled way. Our framework can inspire a variety of neural architectures including some renowned structures such as ResNet and DenseNet, as well as novel innovations. The experimental results on image classification problems demonstrate that the newly inspired networks are competitive and surpass the baselines of ResNet, DenseNet, as well as the advanced ConvNeXt and ViT, testifying to the effectiveness of our framework.

Applying Super-Resolution and Tomography Concepts to Identify Receptive Field Subunits in the Retina

Spatially nonlinear stimulus integration by retinal ganglion cells lies at the heart of various computations performed by the retina. It arises from the nonlinear transmission of signals that ganglion cells receive from bipolar cells, which thereby constitute functional subunits within a ganglion cell's receptive field. Inferring these subunits from recorded ganglion cell activity promises a new avenue for studying the functional architecture of the retina. This calls for efficient methods, which leave sufficient experimental time to leverage the acquired knowledge for further investigating identified subunits. Here, we combine concepts from super-resolution microscopy and computed tomography and introduce super-resolved tomographic reconstruction (STR) as a technique to efficiently stimulate and locate receptive field subunits. Simulations demonstrate that this approach can reliably identify subunits across a wide range of model variations, and application in recordings of primate parasol ganglion cells validates the experimental feasibility. STR can potentially reveal comprehensive subunit layouts within only a few tens of minutes of recording time, making it ideal for online analysis and closed-loop investigations of receptive field substructure in retina recordings.

Linear diffusion noise boosted deep image prior for unsupervised sparse-view CT reconstruction

Objective.Deep learning has markedly enhanced the performance of sparse-view computed tomography reconstruction. However, the dependence of these methods on supervised training using high-quality paired datasets, and the necessity for retraining under varied physical acquisition conditions, constrain their generalizability across new imaging contexts and settings.Approach.To overcome these limitations, we propose an unsupervised approach grounded in the deep image prior framework. Our approach advances beyond the conventional single noise level input by incorporating multi-level linear diffusion noise, significantly mitigating the risk of overfitting. Furthermore, we embed non-local self-similarity as a deep implicit prior within a self-attention network structure, improving the model's capability to identify and utilize repetitive patterns throughout the image. Additionally, leveraging imaging physics, gradient backpropagation is performed between the image domain and projection data space to optimize network weights.Main Results.Evaluations with both simulated and clinical cases demonstrate our method's effective zero-shot adaptability across various projection views, highlighting its robustness and flexibility. Additionally, our approach effectively eliminates noise and streak artifacts while significantly restoring intricate image details.Significance. Our method aims to overcome the limitations in current supervised deep learning-based sparse-view CT reconstruction, offering improved generalizability and adaptability without the need for extensive paired training data.

Learning a stable approximation of an existing but unknown inverse mapping: application to the half-time circular Radon transform

Supervised deep learning-based methods have inspired a new wave of image reconstruction methods that implicitly learn effective regularization strategies from a set of training data. While they hold potential for improving image quality, they have also raised concerns regarding their robustness. Instabilities can manifest when learned methods are applied to find approximate solutions to ill-posed image reconstruction problems for which a unique and stable inverse mapping does not exist, which is a typical use case. In this study, we investigate the performance of supervised deep learning-based image reconstruction in an alternate use case in which a stable inverse mapping is known to exist but is not yet analytically available in closed form. For such problems, a deep learning-based method can learn a stable approximation of the unknown inverse mapping that generalizes well to data that differ significantly from the training set. The learned approximation of the inverse mapping eliminates the need to employ an implicit (optimization-based) reconstruction method and can potentially yield insights into the unknown analytic inverse formula. The specific problem addressed is image reconstruction from a particular case of radially truncated circular Radon transform (CRT) data, referred to as 'half-time' measurement data. For the half-time image reconstruction problem, we develop and investigate a learned filtered backprojection method that employs a convolutional neural network to approximate the unknown filtering operation. We demonstrate that this method behaves stably and readily generalizes to data that differ significantly from training data. The developed method may find application to wave-based imaging modalities that include photoacoustic computed tomography.

Structure-aware diffusion for low-dose CT imaging

Reducing the radiation dose leads to the x-ray computed tomography (CT) images suffering from heavy noise and artifacts, which inevitably interferes with the subsequent clinic diagnostic and analysis. Leading works have explored diffusion models for low-dose CT imaging to avoid the structure degeneration and blurring effects of previous deep denoising models. However, most of them always begin their generative processes with Gaussian noise, which has little or no structure priors of the clean data distribution, thereby leading to long-time inference and unpleasant reconstruction quality. To alleviate these problems, this paper presents a Structure-Aware Diffusion model (SAD), an end-to-end self-guided learning framework for high-fidelity CT image reconstruction. First, SAD builds a nonlinear diffusion bridge between clean and degraded data distributions, which could directly learn the implicit physical degradation prior from observed measurements. Second, SAD integrates the prompt learning mechanism and implicit neural representation into the diffusion process, where rich and diverse structure representations extracted by degraded inputs are exploited as prompts, which provides global and local structure priors, to guide CT image reconstruction. Finally, we devise an efficient self-guided diffusion architecture using an iterative updated strategy, which further refines structural prompts during each generative step to drive finer image reconstruction. Extensive experiments on AAPM-Mayo and LoDoPaB-CT datasets demonstrate that our SAD could achieve superior performance in terms of noise removal, structure preservation, and blind-dose generalization, with few generative steps, even one step only.

One-step inverse generation network for sparse-view dual-energy CT reconstruction and material imaging

Objective.Sparse-view dual-energy spectral computed tomography (DECT) imaging is a challenging inverse problem. Due to the incompleteness of the collected data, the presence of streak artifacts can result in the degradation of reconstructed spectral images. The subsequent material decomposition task in DECT can further lead to the amplification of artifacts and noise.Approach.To address this problem, we propose a novel one-step inverse generation network (OIGN) for sparse-view dual-energy CT imaging, which can achieve simultaneous imaging of spectral images and materials. The entire OIGN consists of five sub-networks that form four modules, including the pre-reconstruction module, the pre-decomposition module, and the following residual filtering module and residual decomposition module. The residual feedback mechanism is introduced to synchronize the optimization of spectral CT images and materials.Main results.Numerical simulation experiments show that the OIGN has better performance on both reconstruction and material decomposition than other state-of-the-art spectral CT imaging algorithms. OIGN also demonstrates high imaging efficiency by completing two high-quality imaging tasks in just 50 seconds. Additionally, anti-noise testing is conducted to evaluate the robustness of OIGN.Significance.These findings have great potential in high-quality multi-task spectral CT imaging in clinical diagnosis.

Improving quantitative MRI using self-supervised deep learning with model reinforcement: Demonstration for rapid T1 mapping

PURPOSE

This paper proposes a novel self-supervised learning framework that uses model reinforcement, REference-free LAtent map eXtraction with MOdel REinforcement (RELAX-MORE), for accelerated quantitative MRI (qMRI) reconstruction. The proposed method uses an optimization algorithm to unroll an iterative model-based qMRI reconstruction into a deep learning framework, enabling accelerated MR parameter maps that are highly accurate and robust.

METHODS

Unlike conventional deep learning methods which require large amounts of training data, RELAX-MORE is a subject-specific method that can be trained on single-subject data through self-supervised learning, making it accessible and practically applicable to many qMRI studies. Using quantitativeT 1 $$ {\mathrm{T}}_1 $$ mapping as an example, the proposed method was applied to the brain, knee and phantom data.

RESULTS

The proposed method generates high-quality MR parameter maps that correct for image artifacts, removes noise, and recovers image features in regions of imperfect image conditions. Compared with other state-of-the-art conventional and deep learning methods, RELAX-MORE significantly improves efficiency, accuracy, robustness, and generalizability for rapid MR parameter mapping.

CONCLUSION

This work demonstrates the feasibility of a new self-supervised learning method for rapid MR parameter mapping, that is readily adaptable to the clinical translation of qMRI.

DDT-Net: Dose-Agnostic Dual-Task Transfer Network for Simultaneous Low-Dose CT Denoising and Simulation

Deep learning (DL) algorithms have achieved unprecedented success in low-dose CT (LDCT) imaging and are expected to be a new generation of CT reconstruction technology. However, most DL-based denoising models often lack the ability to generalize to unseen dose data. Moreover, most simulation tools for LDCT typically operate on proprietary projection data, which is generally not accessible without an established collaboration with CT manufacturers. To alleviate these issues, in this work, we propose a dose-agnostic dual-task transfer network, termed DDT-Net, for simultaneous LDCT denoising and simulation. Concretely, the dual-task learning module is constructed to integrate the LDCT denoising and simulation tasks into a unified optimization framework by learning the joint distribution of LDCT and NDCT data. We approximate the joint distribution of continuous dose level data by training DDT-Net with discrete dose data, which can be generalized to denoising and simulation of unseen dose data. In particular, the mixed-dose training strategy adopted by DDT-Net can promote the denoising performance of lower-dose data. The paired dataset simulated by DDT-Net can be used for data augmentation to further restore the tissue texture of LDCT images. Experimental results on synthetic data and clinical data show that the proposed DDT-Net outperforms competing methods in terms of denoising and generalization performance at unseen dose data, and it also provides a simulation tool that can quickly simulate realistic LDCT images at arbitrary dose levels.

Noise-Generating and Imaging Mechanism Inspired Implicit Regularization Learning Network for Low Dose CT Reconstrution

Low-dose computed tomography (LDCT) helps to reduce radiation risks in CT scanning while maintaining image quality, which involves a consistent pursuit of lower incident rays and higher reconstruction performance. Although deep learning approaches have achieved encouraging success in LDCT reconstruction, most of them treat the task as a general inverse problem in either the image domain or the dual (sinogram and image) domains. Such frameworks have not considered the original noise generation of the projection data and suffer from limited performance improvement for the LDCT task. In this paper, we propose a novel reconstruction model based on noise-generating and imaging mechanism in full-domain, which fully considers the statistical properties of intrinsic noises in LDCT and prior information in sinogram and image domains. To solve the model, we propose an optimization algorithm based on the proximal gradient technique. Specifically, we derive the approximate solutions of the integer programming problem on the projection data theoretically. Instead of hand-crafting the sinogram and image regularizers, we propose to unroll the optimization algorithm to be a deep network. The network implicitly learns the proximal operators of sinogram and image regularizers with two deep neural networks, providing a more interpretable and effective reconstruction procedure. Numerical results demonstrate our proposed method improvements of > 2.9 dB in peak signal to noise ratio, > 1.4% promotion in structural similarity metric, and > 9 HU decrements in root mean square error over current state-of-the-art LDCT methods.

RT-SRTS: Angle-agnostic real-time simultaneous 3D reconstruction and tumor segmentation from single X-ray projection

Radiotherapy is one of the primary treatment methods for tumors, but the organ movement caused by respiration limits its accuracy. Recently, 3D imaging from a single X-ray projection has received extensive attention as a promising approach to address this issue. However, current methods can only reconstruct 3D images without directly locating the tumor and are only validated for fixed-angle imaging, which fails to fully meet the requirements of motion control in radiotherapy. In this study, a novel imaging method RT-SRTS is proposed which integrates 3D imaging and tumor segmentation into one network based on multi-task learning (MTL) and achieves real-time simultaneous 3D reconstruction and tumor segmentation from a single X-ray projection at any angle. Furthermore, the attention enhanced calibrator (AEC) and uncertain-region elaboration (URE) modules have been proposed to aid feature extraction and improve segmentation accuracy. The proposed method was evaluated on fifteen patient cases and compared with three state-of-the-art methods. It not only delivers superior 3D reconstruction but also demonstrates commendable tumor segmentation results. Simultaneous reconstruction and segmentation can be completed in approximately 70 ms, significantly faster than the required time threshold for real-time tumor tracking. The efficacies of both AEC and URE have also been validated in ablation studies. The code of work is available at https://github.com/ZywooSimple/RT-SRTS.

Image Reconstruction for Accelerated MR Scan With Faster Fourier Convolutional Neural Networks

High quality image reconstruction from undersampled k -space data is key to accelerating MR scanning. Current deep learning methods are limited by the small receptive fields in reconstruction networks, which restrict the exploitation of long-range information, and impede the mitigation of full-image artifacts, particularly in 3D reconstruction tasks. Additionally, the substantial computational demands of 3D reconstruction considerably hinder advancements in related fields. To tackle these challenges, we propose the following: 1) A novel convolution operator named Faster Fourier Convolution (FasterFC), aims at providing an adaptable broad receptive field for spatial domain reconstruction networks with fast computational speed. 2) A split-slice strategy that substantially reduces the computational load of 3D reconstruction, enabling high-resolution, multi-coil, 3D MR image reconstruction while fully utilizing inter-layer and intra-layer information. 3) A single-to-group algorithm that efficiently utilizes scan-specific and data-driven priors to enhance k -space interpolation effects. 4) A multi-stage, multi-coil, 3D fast MRI method, called the faster Fourier convolution based single-to-group network (FAS-Net), comprising a single-to-group k -space interpolation algorithm and a FasterFC-based image domain reconstruction module, significantly minimizes the computational demands of 3D reconstruction through split-slice strategy. Experimental evaluations conducted on the NYU fastMRI and Stanford MRI Data datasets reveal that the FasterFC significantly enhances the quality of both 2D and 3D reconstruction results. Moreover, FAS-Net, characterized as a method that can achieve high-resolution (320, 320, 256), multi-coil, (8 coils), 3D fast MRI, exhibits superior reconstruction performance compared to other state-of-the-art 2D and 3D methods.

Joint reconstruction and segmentation in undersampled 3D knee MRI combining shape knowledge and deep learning

Objective.Task-adapted image reconstruction methods using end-to-end trainable neural networks (NNs) have been proposed to optimize reconstruction for subsequent processing tasks, such as segmentation. However, their training typically requires considerable hardware resources and thus, only relatively simple building blocks, e.g. U-Nets, are typically used, which, albeit powerful, do not integrate model-specific knowledge.Approach.In this work, we extend an end-to-end trainable task-adapted image reconstruction method for a clinically realistic reconstruction and segmentation problem of bone and cartilage in 3D knee MRI by incorporating statistical shape models (SSMs). The SSMs model the prior information and help to regularize the segmentation maps as a final post-processing step. We compare the proposed method to a simultaneous multitask learning approach for image reconstruction and segmentation (MTL) and to a complex SSMs-informed segmentation pipeline (SIS).Main results.Our experiments show that the combination of joint end-to-end training and SSMs to further regularize the segmentation maps obtained by MTL highly improves the results, especially in terms of mean and maximal surface errors. In particular, we achieve the segmentation quality of SIS and, at the same time, a substantial model reduction that yields a five-fold decimation in model parameters and a computational speedup of an order of magnitude.Significance.Remarkably, even for undersampling factors of up toR= 8, the obtained segmentation maps are of comparable quality to those obtained by SIS from ground-truth images.

Semi-supervised iterative adaptive network for low-dose CT sinogram recovery

Background.Concern has been expressed regarding the risk of carcinogenesis from medical computed tomography (CT) radiation. Lowering radiation in CT without appropriate modifications often leads to severe noise-induced artifacts in the images. The utilization of deep learning (DL) techniques has achieved promising reconstruction performance in low-dose CT (LDCT) imaging. However, most DL-based algorithms require the pre-collection of a large set of image pairs (low-dose/standard-dose) and the training of networks in an end-to-end supervised manner. Meanwhile, securing such a large volume of paired, well-registered training data in clinical practice is challenging. Moreover, these algorithms often overlook the potential to utilize the abundant information in a large collection of LDCT-only images/sinograms.Methods.In this paper, we introduce a semi-supervised iterative adaptive network (SIA-Net) for LDCT imaging, utilizing both labeled and unlabeled sinograms in a cohesive network framework, integrating supervised and unsupervised learning processes. Specifically, the supervised process captures critical features (i.e. noise distribution and tissue characteristics) latent in the paired sinograms, while the unsupervised process effectively learns these features in the unlabeled low-dose sinograms, employing a conventional weighted least-squares model with a regularization term. Furthermore, the SIA-Net method is designed to adaptively transfer the learned feature distribution from the supervised to the unsupervised process, thereby obtaining a high-fidelity sinogram through iterative adaptive learning. Finally, high-quality CT images can be reconstructed from the refined sinogram using the filtered back-projection algorithm.Results.Experimental results on two clinical datasets indicate that the proposed SIA-Net method achieves competitive performance in terms of noise reduction and structure preservation in LDCT imaging, when compared to traditional supervised learning methods.

DCT-net: Dual-domain cross-fusion transformer network for MRI reconstruction

Current challenges in Magnetic Resonance Imaging (MRI) include long acquisition times and motion artifacts. To address these issues, under-sampled k-space acquisition has gained popularity as a fast imaging method. However, recovering fine details from under-sampled data remains challenging. In this study, we introduce a pioneering deep learning approach, namely DCT-Net, designed for dual-domain MRI reconstruction. DCT-Net seamlessly integrates information from the image domain (IRM) and frequency domain (FRM), utilizing a novel Cross Attention Block (CAB) and Fusion Attention Block (FAB). These innovative blocks enable precise feature extraction and adaptive fusion across both domains, resulting in a significant enhancement of the reconstructed image quality. The adaptive interaction and fusion mechanisms of CAB and FAB contribute to the method's effectiveness in capturing distinctive features and optimizing image reconstruction. Comprehensive ablation studies have been conducted to assess the contributions of these modules to reconstruction quality and accuracy. Experimental results on the FastMRI (2023) and Calgary-Campinas datasets (2021) demonstrate the superiority of our MRI reconstruction framework over other typical methods (most are illustrated in 2023 or 2022) in both qualitative and quantitative evaluations. This holds for knee and brain datasets under 4× and 8× accelerated imaging scenarios.

A primal-dual data-driven method for computational optical imaging with a photonic lantern

Optical fibers aim to image in vivo biological processes. In this context, high spatial resolution and stability to fiber movements are key to enable decision-making processes (e.g. for microendoscopy). Recently, a single-pixel imaging technique based on a multicore fiber photonic lantern has been designed, named computational optical imaging using a lantern (COIL). A proximal algorithm based on a sparsity prior, dubbed SARA-COIL, has been further proposed to solve the associated inverse problem, to enable image reconstructions for high resolution COIL microendoscopy. In this work, we develop a data-driven approach for COIL. We replace the sparsity prior in the proximal algorithm by a learned denoiser, leading to a plug-and-play (PnP) algorithm. The resulting PnP method, based on a proximal primal-dual algorithm, enables to solve the Morozov formulation of the inverse problem. We use recent results in learning theory to train a network with desirable Lipschitz properties, and we show that the resulting primal-dual PnP algorithm converges to a solution to a monotone inclusion problem. Our simulations highlight that the proposed data-driven approach improves the reconstruction quality over variational SARA-COIL method on both simulated and real data.

SWISTA-Nets: Subband-adaptive wavelet iterative shrinkage thresholding networks for image reconstruction

Robust and interpretable image reconstruction is central to imageology applications in clinical practice. Prevalent deep networks, with strong learning ability to extract implicit information from data manifold, are still lack of prior knowledge introduced from mathematics or physics, leading to instability, poor structure interpretability and high computation cost. As to this issue, we propose two prior knowledge-driven networks to combine the good interpretability of mathematical methods and the powerful learnability of deep learning methods. Incorporating different kinds of prior knowledge, we propose subband-adaptive wavelet iterative shrinkage thresholding networks (SWISTA-Nets), where almost every network module is in one-to-one correspondence with each step involved in the iterative algorithm. By end-to-end training of proposed SWISTA-Nets, implicit information can be extracted from training data and guide the tuning process of key parameters that possess mathematical definition. The inverse problems associated with two medical imaging modalities, i.e., electromagnetic tomography and X-ray computational tomography are applied to validate the proposed networks. Both visual and quantitative results indicate that the SWISTA-Nets outperform mathematical methods and state-of-the-art prior knowledge-driven networks, especially with fewer training parameters, interpretable network structures and well robustness. We assume that our analysis will support further investigation of prior knowledge-driven networks in the field of ill-posed image reconstruction.

AIRPORT: A Data Consistency Constrained Deep Temporal Extrapolation Method To Improve Temporal Resolution In Contrast Enhanced CT Imaging

Typical tomographic image reconstruction methods require that the imaged object is static and stationary during the time window to acquire a minimally complete data set. The violation of this requirement leads to temporal-averaging errors in the reconstructed images. For a fixed gantry rotation speed, to reduce the errors, it is desired to reconstruct images using data acquired over a narrower angular range, i.e., with a higher temporal resolution. However, image reconstruction with a narrower angular range violates the data sufficiency condition, resulting in severe data-insufficiency-induced errors. The purpose of this work is to decouple the trade-off between these two types of errors in contrast-enhanced computed tomography (CT) imaging. We demonstrated that using the developed data consistency constrained deep temporal extrapolation method (AIRPORT), the entire time-varying imaged object can be accurately reconstructed with 40 frames-per-second temporal resolution, the time window needed to acquire a single projection view data using a typical C-arm cone-beam CT system. AIRPORT is applicable to general non-sparse imaging tasks using a single short-scan data acquisition.

Proj2Proj: self-supervised low-dose CT reconstruction

In Computed Tomography (CT) imaging, one of the most serious concerns has always been ionizing radiation. Several approaches have been proposed to reduce the dose level without compromising the image quality. With the emergence of deep learning, thanks to the increasing availability of computational power and huge datasets, data-driven methods have recently received a lot of attention. Deep learning based methods have also been applied in various ways to address the low-dose CT reconstruction problem. However, the success of these methods largely depends on the availability of labeled data. On the other hand, recent studies showed that training can be done successfully without the need for labeled datasets. In this study, a training scheme was defined to use low-dose projections as their own training targets. The self-supervision principle was applied in the projection domain. The parameters of a denoiser neural network were optimized through self-supervised training. It was shown that our method outperformed both traditional and compressed sensing-based iterative methods, and deep learning based unsupervised methods, in the reconstruction of analytic CT phantoms and human CT images in low-dose CT imaging. Our method's reconstruction quality is also comparable to a well-known supervised method.

Deep convolutional dictionary learning network for sparse view CT reconstruction with a group sparse prior

Purpose Numerous techniques based on deep learning have been utilized in sparse view computed tomography (CT) imaging. Nevertheless, the majority of techniques are instinctively constructed utilizing state-of-the-art opaque convolutional neural networks (CNNs) and lack interpretability. Moreover, CNNs tend to focus on local receptive fields and neglect nonlocal self-similarity prior information. Obtaining diagnostically valuable images from sparsely sampled projections is a challenging and ill-posed task. Method To address this issue, we propose a unique and understandable model named DCDL-GS for sparse view CT imaging. This model relies on a network comprised of convolutional dictionary learning and a nonlocal group sparse prior. To enhance the quality of image reconstruction, we utilize a neural network in conjunction with a statistical iterative reconstruction framework and perform a set number of iterations. Inspired by group sparsity priors, we adopt a novel group thresholding operation to improve the feature representation and constraint ability and obtain a theoretical interpretation. Furthermore, our DCDL-GS model incorporates filtered backprojection (FBP) reconstruction, fast sliding window nonlocal self-similarity operations, and a lightweight and interpretable convolutional dictionary learning network to enhance the applicability of the model. Results The efficiency of our proposed DCDL-GS model in preserving edges and recovering features is demonstrated by the visual results obtained on the LDCT-P and UIH datasets. Compared to the results of the most advanced techniques, the quantitative results are enhanced, with increases of 0.6-0.8 dB for the peak signal-to-noise ratio (PSNR), 0.005-0.01 for the structural similarity index measure (SSIM), and 1-1.3 for the regulated Fréchet inception distance (rFID) on the test dataset. The quantitative results also show the effectiveness of our proposed deep convolution iterative reconstruction module and nonlocal group sparse prior. Conclusion In this paper, we create a consolidated and enhanced mathematical model by integrating projection data and prior knowledge of images into a deep iterative model. The model is more practical and interpretable than existing approaches. The results from the experiment show that the proposed model performs well in comparison to the others.

Systematic Review on Learning-based Spectral CT

Spectral computed tomography (CT) has recently emerged as an advanced version of medical CT and significantly improves conventional (single-energy) CT. Spectral CT has two main forms: dual-energy computed tomography (DECT) and photon-counting computed tomography (PCCT), which offer image improvement, material decomposition, and feature quantification relative to conventional CT. However, the inherent challenges of spectral CT, evidenced by data and image artifacts, remain a bottleneck for clinical applications. To address these problems, machine learning techniques have been widely applied to spectral CT. In this review, we present the state-of-the-art data-driven techniques for spectral CT.

Deep Filtered Back Projection for CT Reconstruction

Filtered back projection (FBP) is a classic analytical algorithm for computed tomography (CT) reconstruction, with high computational efficiency. However, images reconstructed by FBP often suffer from excessive noise and artifacts. The original FBP algorithm uses a window function to smooth signals and a linear interpolation to estimate projection values at un-sampled locations. In this study, we propose a novel framework named DeepFBP in which an optimized filter and an optimized nonlinear interpolation operator are learned with neural networks. Specifically, the learned filter can be considered as the product of an optimized window function and the ramp filter, and the learned interpolation can be considered as an optimized way to utilize projection information of nearby locations through nonlinear combination. The proposed method remains the high computational efficiency of the original FBP and achieves much better reconstruction quality at different noise levels. It also outperforms the TV-based statistical iterative algorithm, with computational time being reduced in an order of two, and state-of-the-art post-processing deep learning methods that have deeper and more complicated network structures.

PARALLELPROJ-an open-source framework for fast calculation of projections in tomography

In this article, we introduce parallelproj, a novel open-source framework designed for efficient parallel computation of projections in tomography leveraging either multiple CPU cores or GPUs. This framework efficiently implements forward and back projection functions for both sinogram and listmode data, utilizing Joseph's method, which is further extended to encompass time-of-flight (TOF) PET projections. Our evaluation involves a series of tests focusing on PET image reconstruction using data sourced from a state-of-the-art clinical PET/CT system. We thoroughly benchmark the performance of the projectors in non-TOF and TOF, sinogram, and listmode employing multi CPU-cores, hybrid CPU/GPU, and exclusive GPU mode. Moreover, we also investigate the timing of non-TOF sinogram projections calculated in STIR (Software for Tomographic Image Reconstruction) which recently integrated parallelproj as one of its projection backends. Our results indicate that the exclusive GPU mode provides acceleration factors between 25 and 68 relative to the multi-CPU-core mode. Furthermore, we demonstrate that OSEM listmode reconstruction of state-of-the-art real-world PET data sets is achievable within a few seconds using a single consumer GPU.

"Under the hood": artificial intelligence in personalized radiotherapy

This review presents and discusses the ways in which artificial intelligence (AI) tools currently intervene, or could potentially intervene in the future, to enhance the diverse tasks involved in the radiotherapy workflow. The radiotherapy framework is presented on 2 different levels for the personalization of the treatment, distinct in tasks and methodologies. The first level is the clinically well-established anatomy-based workflow, known as adaptive radiation therapy. The second level is referred to as biology-driven workflow, explored in the research literature and recently appearing in some preliminary clinical trials for personalized radiation treatments. A 2-fold role for AI is defined according to these 2 different levels. In the anatomy-based workflow, the role of AI is to streamline and improve the tasks in terms of time and variability reductions compared to conventional methodologies. The biology-driven workflow instead fully relies on AI, which introduces decision-making tools opening uncharted frontiers that were in the past deemed challenging to explore. These methodologies are referred to as radiomics and dosiomics, handling imaging and dosimetric information, or multiomics, when complemented by clinical and biological parameters (ie, biomarkers). The review explicitly highlights the methodologies that are currently incorporated into clinical practice or still in research, with the aim of presenting the AI's growing role in personalized radiotherapy.

Deep Generalized Learning Model for PET Image Reconstruction

Low-count positron emission tomography (PET) imaging is challenging because of the ill-posedness of this inverse problem. Previous studies have demonstrated that deep learning (DL) holds promise for achieving improved low-count PET image quality. However, almost all data-driven DL methods suffer from fine structure degradation and blurring effects after denoising. Incorporating DL into the traditional iterative optimization model can effectively improve its image quality and recover fine structures, but little research has considered the full relaxation of the model, resulting in the performance of this hybrid model not being sufficiently exploited. In this paper, we propose a learning framework that deeply integrates DL and an alternating direction of multipliers method (ADMM)-based iterative optimization model. The innovative feature of this method is that we break the inherent forms of the fidelity operators and use neural networks to process them. The regularization term is deeply generalized. The proposed method is evaluated on simulated data and real data. Both the qualitative and quantitative results show that our proposed neural network method can outperform partial operator expansion-based neural network methods, neural network denoising methods and traditional methods.

A Deep Convolutional Gated Recurrent Unit for CT Image Reconstruction

Computed tomography (CT) is one of the most important medical imaging technologies in use today. Most commercial CT products use a technique known as the filtered backprojection (FBP) that is fast and can produce decent image quality when an X-ray dose is high. However, the FBP is not good enough on low-dose X-ray CT imaging because the CT image reconstruction problem becomes more stochastic. A more effective reconstruction technique proposed recently and implemented in a limited number of CT commercial products is an iterative reconstruction (IR). The IR technique is based on a Bayesian formulation of the CT image reconstruction problem with an explicit model of the CT scanning, including its stochastic nature, and a prior model that incorporates our knowledge about what a good CT image should look like. However, constructing such prior knowledge is more complicated than it seems. In this article, we propose a novel neural network for CT image reconstruction. The network is based on the IR formulation and constructed with a recurrent neural network (RNN). Specifically, we transform the gated recurrent unit (GRU) into a neural network performing CT image reconstruction. We call it "GRU reconstruction." This neural network conducts concurrent dual-domain learning. Many deep learning (DL)-based methods in medical imaging are single-domain learning, but dual-domain learning performs better because it learns from both the sinogram and the image domain. In addition, we propose backpropagation through stage (BPTS) as a new RNN backpropagation algorithm. It is similar to the backpropagation through time (BPTT) of an RNN; however, it is tailored for iterative optimization. Results from extensive experiments indicate that our proposed method outperforms conventional model-based methods, single-domain DL methods, and state-of-the-art DL techniques in terms of the root mean squared error (RMSE), the peak signal-to-noise ratio (PSNR), and the structure similarity (SSIM) and in terms of visual appearance.

Equilibrated Zeroth-Order Unrolled Deep Network for Parallel MR Imaging

In recent times, model-driven deep learning has evolved an iterative algorithm into a cascade network by replacing the regularizer's first-order information, such as the (sub)gradient or proximal operator, with a network module. This approach offers greater explainability and predictability compared to typical data-driven networks. However, in theory, there is no assurance that a functional regularizer exists whose first-order information matches the substituted network module. This implies that the unrolled network output may not align with the regularization models. Furthermore, there are few established theories that guarantee global convergence and robustness (regularity) of unrolled networks under practical assumptions. To address this gap, we propose a safeguarded methodology for network unrolling. Specifically, for parallel MR imaging, we unroll a zeroth-order algorithm, where the network module serves as a regularizer itself, allowing the network output to be covered by a regularization model. Additionally, inspired by deep equilibrium models, we conduct the unrolled network before backpropagation to converge to a fixed point and then demonstrate that it can tightly approximate the actual MR image. We also prove that the proposed network is robust against noisy interferences if the measurement data contain noise. Finally, numerical experiments indicate that the proposed network consistently outperforms state-of-the-art MRI reconstruction methods, including traditional regularization and unrolled deep learning techniques.

End-to-end memory-efficient reconstruction for cone beam CT

BACKGROUND

Cone beam computed tomography (CBCT) plays an important role in many medical fields nowadays. Unfortunately, the potential of this imaging modality is hampered by lower image quality compared to the conventional CT, and producing accurate reconstructions remains challenging. A lot of recent research has been directed towards reconstruction methods relying on deep learning, which have shown great promise for various imaging modalities. However, practical application of deep learning to CBCT reconstruction is complicated by several issues, such as exceedingly high memory costs of deep learning methods when working with fully 3D data. Additionally, deep learning methods proposed in the literature are often trained and evaluated only on data from a specific region of interest, thus raising concerns about possible lack of generalization to other regions.

PURPOSE

In this work, we aim to address these limitations and propose LIRE: a learned invertible primal-dual iterative scheme for CBCT reconstruction.

METHODS

LIRE is a learned invertible primal-dual iterative scheme for CBCT reconstruction, wherein we employ a U-Net architecture in each primal block and a residual convolutional neural network (CNN) architecture in each dual block. Memory requirements of the network are substantially reduced while preserving its expressive power through a combination of invertible residual primal-dual blocks and patch-wise computations inside each of the blocks during both forward and backward pass. These techniques enable us to train on data with isotropic 2 mm voxel spacing, clinically-relevant projection count and detector panel resolution on current hardware with 24 GB video random access memory (VRAM).

RESULTS

Two LIRE models for small and for large field-of-view (FoV) setting were trained and validated on a set of 260 + 22 thorax CT scans and tested using a set of 142 thorax CT scans plus an out-of-distribution dataset of 79 head and neck CT scans. For both settings, our method surpasses the classical methods and the deep learning baselines on both test sets. On the thorax CT set, our method achieves peak signal-to-noise ratio (PSNR) of 33.84 ± 2.28 for the small FoV setting and 35.14 ± 2.69 for the large FoV setting; U-Net baseline achieves PSNR of 33.08 ± 1.75 and 34.29 ± 2.71 respectively. On the head and neck CT set, our method achieves PSNR of 39.35 ± 1.75 for the small FoV setting and 41.21 ± 1.41 for the large FoV setting; U-Net baseline achieves PSNR of 33.08 ± 1.75 and 34.29 ± 2.71 respectively. Additionally, we demonstrate that LIRE can be finetuned to reconstruct high-resolution CBCT data with the same geometry but 1 mm voxel spacing and higher detector panel resolution, where it outperforms the U-Net baseline as well.

CONCLUSIONS

Learned invertible primal-dual schemes with additional memory optimizations can be trained to reconstruct CBCT volumes directly from the projection data with clinically-relevant geometry and resolution. Such methods can offer better reconstruction quality and generalization compared to classical deep learning baselines.

Noise2Recon: Enabling SNR-robust MRI reconstruction with semi-supervised and self-supervised learning

PURPOSE

To develop a method for building MRI reconstruction neural networks robust to changes in signal-to-noise ratio (SNR) and trainable with a limited number of fully sampled scans.

METHODS

We propose Noise2Recon, a consistency training method for SNR-robust accelerated MRI reconstruction that can use both fully sampled (labeled) and undersampled (unlabeled) scans. Noise2Recon uses unlabeled data by enforcing consistency between model reconstructions of undersampled scans and their noise-augmented counterparts. Noise2Recon was compared to compressed sensing and both supervised and self-supervised deep learning baselines. Experiments were conducted using retrospectively accelerated data from the mridata three-dimensional fast-spin-echo knee and two-dimensional fastMRI brain datasets. All methods were evaluated in label-limited settings and among out-of-distribution (OOD) shifts, including changes in SNR, acceleration factors, and datasets. An extensive ablation study was conducted to characterize the sensitivity of Noise2Recon to hyperparameter choices.

RESULTS

In label-limited settings, Noise2Recon achieved better structural similarity, peak signal-to-noise ratio, and normalized-RMS error than all baselines and matched performance of supervised models, which were trained with14 × $$ 14\times $$ more fully sampled scans. Noise2Recon outperformed all baselines, including state-of-the-art fine-tuning and augmentation techniques, among low-SNR scans and when generalizing to OOD acceleration factors. Augmentation extent and loss weighting hyperparameters had negligible impact on Noise2Recon compared to supervised methods, which may indicate increased training stability.

CONCLUSION

Noise2Recon is a label-efficient reconstruction method that is robust to distribution shifts, such as changes in SNR, acceleration factors, and others, with limited or no fully sampled training data.