PET Image Reconstruction Using Deep Image Prior

The Evolution of Artificial Intelligence in Nuclear Medicine

Nuclear medicine has continuously evolved since its beginnings, constantly improving the diagnosis and treatment of various diseases. The integration of artificial intelligence (AI) is one of the latest revolutionizing chapters, promising significant advancements in diagnosis, prognosis, segmentation, image quality enhancement, and theranostics. Early AI applications in nuclear medicine focused on improving diagnostic accuracy, leveraging machine learning algorithms for disease classification and outcome prediction. Advances in deep learning, including convolutional and more recently transformer-based neural networks, have further enabled more precise diagnosis and image segmentation as well as low-dose imaging, and patient-specific dosimetry for personalized treatment. Generative AI, driven by large language models and diffusion techniques, is now allowing the process, interpretation, and generation of complex medical language and images. Despite these achievements, challenges such as data scarcity, heterogeneity, and ethical concerns remain barriers to clinical translation. Addressing these issues through interdisciplinary collaboration will pave the way for a broader adoption of AI in nuclear medicine, potentially enhancing patient care and optimizing diagnosis and therapeutic outcomes.

RED: Residual estimation diffusion for low-dose PET sinogram reconstruction

Recent advances in diffusion models have demonstrated exceptional performance in generative tasks across various fields. In positron emission tomography (PET), the reduction in tracer dose leads to information loss in sinograms. Using diffusion models to reconstruct missing information can improve imaging quality. Traditional diffusion models effectively use Gaussian noise for image reconstructions. However, in low-dose PET reconstruction, Gaussian noise can worsen the already sparse data by introducing artifacts and inconsistencies. To address this issue, we propose a diffusion model named residual estimation diffusion (RED). From the perspective of diffusion mechanism, RED uses the residual between sinograms to replace Gaussian noise in diffusion process, respectively sets the low-dose and full-dose sinograms as the starting point and endpoint of reconstruction. This mechanism helps preserve the original information in the low-dose sinogram, thereby enhancing reconstruction reliability. From the perspective of data consistency, RED introduces a drift correction strategy to reduce accumulated prediction errors during the reverse process. Calibrating the intermediate results of reverse iterations helps maintain the data consistency and enhances the stability of reconstruction process. In the experiments, RED achieved the best performance across all metrics. Specifically, the PSNR metric showed improvements of 2.75, 5.45, and 8.08 dB in DRF4, 20, and 100 respectively, compared to traditional methods. The code is available at: https://github.com/yqx7150/RED.

External phantom-based validation of a deep-learning network trained for upscaling of digital low count PET data

BACKGROUND

A reduction of dose and/or acquisition duration of PET examinations is desirable in terms of radiation protection, patient comfort and throughput, but leads to decreased image quality due to poorer image statistics. Recently, different deep-learning based methods have been proposed to improve image quality of low-count PET images. For example, one such approach allows the generation of AI-enhanced PET images (AI-PET) based on ultra-low count PET/CT scans. The performance of this algorithm has so far only been clinically evaluated on patient data featuring limited scan statistics and unknown actual activity concentration. Therefore, this study investigates the performance of this deep-learning algorithm using PET measurements of a phantom resembling different lesion sizes and count statistics (from ultra-low to high) to understand the capabilities and limitations of AI-based post processing for improved image quality in ultra-low count PET imaging.

METHODS

A previously trained pix2pixHD Generative Adversarial Network was evaluated. To this end, a NEMA PET body phantom filled with two sphere-to-background activity concentration ratios (4:1 and 10:1) and two attenuation scenarios to investigate the effects of obese patients was scanned in list mode. Images were reconstructed with 13 different acquisition durations ranging from 5 s up to 900 s. Image noise, recovery coefficients, SUV-differences, image quality measurement metrics such as the Structural Similarity Index Metric, and the contrast-to-noise-ratio were assessed. In addition, the benefits of the deep-learning network over Gaussian smoothing were investigated.

RESULTS

The presented AI-algorithm is very well suitable for denoising ultra-low count PET images and for restoring structural information, but increases image noise in ultra-high count PET scans. The generated AI-PET scans strongly underestimate SUV especially in small lesions with a diameter ≤ 17 mm, while quantitative measures of large lesions ≥ 37 mm in diameter were accurately recovered. In ultra-low count or low contrast images, the AI algorithm might not be able to recognize small lesions ≤ 13 mm in diameter. In comparison to standardized image post-processing using a Gaussian filter, the deep-learning network is better suited to improve image quality, but at the same time degrades SUV accuracy to a greater extent than post-filtering and quantitative SUV accuracy varies for different lesion sizes.

CONCLUSIONS

Phantom-based validation of AI-based algorithms allows for a detailed assessment of the performance, limitations, and generalizability of deep-learning based algorithms for PET image enhancement. Here it was confirmed that the AI-based approach performs very well in denoising ultra-low count PET images and outperforms traditional Gaussian post-filtering. However, there are strong limitations in terms of quantitative accuracy and detectability of small lesions.

Fundamental study on improving the quality of X-ray fluorescence computed tomography images by applying deep image prior to projection images as a pre-denoising method

PURPOSE

We are developing a three-dimensional X-ray fluorescence computed tomography (3D XFCT) system using non-radioactive-labeled compounds for preclinical studies as a new modality that provides images of biological functions. Improvements in image quality and detection limits are required for the in vivo imaging. The aim of this study was to improve the quality of XFCT images by applying a deep image prior (DIP), which is a type of convolutional neural network, to projection images as a pre-denoising method, and then compare with DIP post-denoising.

METHODS

DIP can restore images using only the projection images acquired by XFCT. The projected images were processed with DIP for denoising. Three-dimensional images were reconstructed using the ordered subsets expectation maximization method for XFCT systems with multi-pinhole collimators. To evaluate the effectiveness of DIP pre-denoising, we constructed an XFCT system using synchrotron radiation and performed imaging experiments on a physical phantom and a mouse brain sample. The proposed method was compared with the DIP post-denoising and other denoising methods.

RESULTS

The proposed DIP pre-denoising method reduced noise and significantly improved the image quality and was superior to the DIP post-denoising and other methods. The contrast-to-noise ratio improved by 3.7 to 4.6 times with almost no deterioration in spatial resolution, and the detection limit improved from 0.069 to 0.035 mg/mL. There was a strong linear relationship between the iodine concentration and pixel values. Finally, image quality of the mouse brain improved.

CONCLUSIONS

Through experiments using phantoms and mouse brains, this study demonstrated that the application of DIP to projection images as a pre-denoising method can significantly improve the image quality and detection limit of 3D XFCT without degrading the spatial resolution. DIP was more effective when applied as pre-denoising than as post-denoising and can contribute to in vivo 3D imaging in the future.

Self-supervised parametric map estimation for multiplexed PET with a deep image prior

Multiplexed positron emission tomography (mPET) imaging allows simultaneous observation of physiological and pathological information from multiple tracers in a single PET scan. Although supervised deep learning has demonstrated superior performance in mPET image separation compared to purely model-based methods, acquiring large amounts of paired single-tracer data and multi-tracer data for training poses a practical challenge and needs extended scan durations for patients. In addition, the generalisation ability of the supervised learning framework is a concern, as the patient being scanned and their tracer kinetics may potentially fall outside the training distribution. In this work, we propose a self-supervised learning framework based on the deep image prior (DIP) for mPET image separation using just one dataset. In particular, we integrate the multi-tracer compartmental model into the DIP framework to estimate the parametric maps of each tracer from the measured dynamic dual-tracer activity images. Consequently, the separated dynamic single-tracer activity images can be recovered from the estimated tracer-specific parametric maps. In the proposed method, dynamic dual-tracer activity images are used as the training label, and the static dual-tracer image (reconstructed from the same patient data from the start to the end of acquisition) is used as the network input. The performance of the proposed method was evaluated on a simulated brain phantom for dynamic dual-tracer [18F]FDG+[11C]MET activity image separation and parametric map estimation. The results demonstrate that the proposed method outperforms the conventional voxel-wise multi-tracer compartmental modeling method (vMTCM) and the two-step method DIP-Dn+vMTCM (where dynamic dual-tracer activity images are first denoised using a U-net within the DIP framework, followed by vMTCM separation) in terms of lower bias and standard deviation in the separated single-tracer images and also for the estimated parametric maps for each tracer, at both voxel and ROI levels.

Enhancing global sensitivity and uncertainty quantification in medical image reconstruction with Monte Carlo arbitrary-masked mamba

Deep learning has been extensively applied in medical image reconstruction, where Convolutional Neural Networks (CNNs) and Vision Transformers (ViTs) represent the predominant paradigms, each possessing distinct advantages and inherent limitations: CNNs exhibit linear complexity with local sensitivity, whereas ViTs demonstrate quadratic complexity with global sensitivity. The emerging Mamba has shown superiority in learning visual representation, which combines the advantages of linear scalability and global sensitivity. In this study, we introduce MambaMIR, an Arbitrary-Masked Mamba-based model with wavelet decomposition for joint medical image reconstruction and uncertainty estimation. A novel Arbitrary Scan Masking (ASM) mechanism "masks out" redundant information to introduce randomness for further uncertainty estimation. Compared to the commonly used Monte Carlo (MC) dropout, our proposed MC-ASM provides an uncertainty map without the need for hyperparameter tuning and mitigates the performance drop typically observed when applying dropout to low-level tasks. For further texture preservation and better perceptual quality, we employ the wavelet transformation into MambaMIR and explore its variant based on the Generative Adversarial Network, namely MambaMIR-GAN. Comprehensive experiments have been conducted for multiple representative medical image reconstruction tasks, demonstrating that the proposed MambaMIR and MambaMIR-GAN outperform other baseline and state-of-the-art methods in different reconstruction tasks, where MambaMIR achieves the best reconstruction fidelity and MambaMIR-GAN has the best perceptual quality. In addition, our MC-ASM provides uncertainty maps as an additional tool for clinicians, while mitigating the typical performance drop caused by the commonly used dropout.

RBP-DIP: Residual back projection with deep image prior for ill-posed CT reconstruction

The success of deep image prior (DIP) in a number of image processing tasks has motivated their application in image reconstruction problems in computed tomography (CT). In this paper, we introduce a residual back projection technique (RBP) that improves the performance of deep image prior framework in iterative CT reconstruction, especially when the reconstruction problem is highly ill-posed. The RBP-DIP framework uses an untrained U-net in conjunction with a novel residual back projection connection to minimize the objective function while improving reconstruction accuracy. In each iteration, the weights of the untrained U-net are optimized, and the output of the U-net in the current iteration is used to update the input of the U-net in the next iteration through the proposed RBP connection. The introduction of the RBP connection strengthens the regularization effects of the DIP framework in the context of iterative CT reconstruction leading to improvements in accuracy. Our experiments demonstrate that the RBP-DIP framework offers improvements over other state-of-the-art conventional IR methods, as well as pre-trained and untrained models with similar network structures under multiple conditions. These improvements are particularly significant in the few-view and limited-angle CT reconstructions, where the corresponding inverse problems are highly ill-posed and the training data is limited. Furthermore, RBP-DIP has the potential for further improvement. Most existing IR algorithms, pre-trained models, and enhancements applicable to the original DIP algorithm can also be integrated into the RBP-DIP framework.

A review of deep learning-based reconstruction methods for accelerated MRI using spatiotemporal and multi-contrast redundancies

Accelerated magnetic resonance imaging (MRI) has played an essential role in reducing data acquisition time for MRI. Acceleration can be achieved by acquiring fewer data points in k-space, which results in various artifacts in the image domain. Conventional reconstruction methods have resolved the artifacts by utilizing multi-coil information, but with limited robustness. Recently, numerous deep learning-based reconstruction methods have been developed, enabling outstanding reconstruction performances with higher acceleration. Advances in hardware and developments of specialized network architectures have produced such achievements. Besides, MRI signals contain various redundant information including multi-coil redundancy, multi-contrast redundancy, and spatiotemporal redundancy. Utilization of the redundant information combined with deep learning approaches allow not only higher acceleration, but also well-preserved details in the reconstructed images. Consequently, this review paper introduces the basic concepts of deep learning and conventional accelerated MRI reconstruction methods, followed by review of recent deep learning-based reconstruction methods that exploit various redundancies. Lastly, the paper concludes by discussing the challenges, limitations, and potential directions of future developments.

PET image reconstruction using weighted nuclear norm maximization and deep learning prior

The ill-posed Positron emission tomography (PET) reconstruction problem usually results in limited resolution and significant noise. Recently, deep neural networks have been incorporated into PET iterative reconstruction framework to improve the image quality. In this paper, we propose a new neural network-based iterative reconstruction method by using weighted nuclear norm (WNN) maximization, which aims to recover the image details in the reconstruction process. The novelty of our method is the application of WNN maximization rather than WNN minimization in PET image reconstruction. Meanwhile, a neural network is used to control the noise originated from WNN maximization. Our method is evaluated on simulated and clinical datasets. The simulation results show that the proposed approach outperforms state-of-the-art neural network-based iterative methods by achieving the best contrast/noise tradeoff with a remarkable contrast improvement on the lesion contrast recovery. The study on clinical datasets also demonstrates that our method can recover lesions of different sizes while suppressing noise in various low-dose PET image reconstruction tasks. Our code is available athttps://github.com/Kuangxd/PETReconstruction.

Deep generative denoising networks enhance quality and accuracy of gated cardiac PET data

BACKGROUND

Cardiac positron emission tomography (PET) can visualize and quantify the molecular and physiological pathways of cardiac function. However, cardiac and respiratory motion can introduce blurring that reduces PET image quality and quantitative accuracy. Dual cardiac- and respiratory-gated PET reconstruction can mitigate motion artifacts but increases noise as only a subset of data are used for each time frame of the cardiac cycle.

AIM

The objective of this study is to create a zero-shot image denoising framework using a conditional generative adversarial networks (cGANs) for improving image quality and quantitative accuracy in non-gated and dual-gated cardiac PET images.

METHODS

Our study included retrospective list-mode data from 40 patients who underwent an 18F-fluorodeoxyglucose (18F-FDG) cardiac PET study. We initially trained and evaluated a 3D cGAN-known as Pix2Pix-on simulated non-gated low-count PET data paired with corresponding full-count target data, and then deployed the model on an unseen test set acquired on the same PET/CT system including both non-gated and dual-gated PET data.

RESULTS

Quantitative analysis demonstrated that the 3D Pix2Pix network architecture achieved significantly (p value<0.05) enhanced image quality and accuracy in both non-gated and gated cardiac PET images. At 5%, 10%, and 15% preserved count statistics, the model increased peak signal-to-noise ratio (PSNR) by 33.7%, 21.2%, and 15.5%, structural similarity index (SSIM) by 7.1%, 3.3%, and 2.2%, and reduced mean absolute error (MAE) by 61.4%, 54.3%, and 49.7%, respectively. When tested on dual-gated PET data, the model consistently reduced noise, irrespective of cardiac/respiratory motion phases, while maintaining image resolution and accuracy. Significant improvements were observed across all gates, including a 34.7% increase in PSNR, a 7.8% improvement in SSIM, and a 60.3% reduction in MAE.

CONCLUSION

The findings of this study indicate that dual-gated cardiac PET images, which often have post-reconstruction artifacts potentially affecting diagnostic performance, can be effectively improved using a generative pre-trained denoising network.

Two-step optimization for accelerating deep image prior-based PET image reconstruction

Deep learning, particularly convolutional neural networks (CNNs), has advanced positron emission tomography (PET) image reconstruction. However, it requires extensive, high-quality training datasets. Unsupervised learning methods, such as deep image prior (DIP), have shown promise for PET image reconstruction. Although DIP-based PET image reconstruction methods demonstrate superior performance, they involve highly time-consuming calculations. This study proposed a two-step optimization method to accelerate end-to-end DIP-based PET image reconstruction and improve PET image quality. The proposed two-step method comprised a pre-training step using conditional DIP denoising, followed by an end-to-end reconstruction step with fine-tuning. Evaluations using Monte Carlo simulation data demonstrated that the proposed two-step method significantly reduced the computation time and improved the image quality, thereby rendering it a practical and efficient approach for end-to-end DIP-based PET image reconstruction.

Modular GAN: positron emission tomography image reconstruction using two generative adversarial networks

Introduction

The reconstruction of PET images involves converting sinograms, which represent the measured counts of radioactive emissions using detector rings encircling the patient, into meaningful images. However, the quality of PET data acquisition is impacted by physical factors, photon count statistics and detector characteristics, which affect the signal-to-noise ratio, resolution and quantitative accuracy of the resulting images. To address these influences, correction methods have been developed to mitigate each of these issues separately. Recently, generative adversarial networks (GANs) based on machine learning have shown promise in learning the complex mapping between acquired PET data and reconstructed tomographic images. This study aims to investigate the properties of training images that contribute to GAN performance when non-clinical images are used for training. Additionally, we describe a method to correct common PET imaging artefacts without relying on patient-specific anatomical images.

Methods

The modular GAN framework includes two GANs. Module 1, resembling Pix2pix architecture, is trained on non-clinical sinogram-image pairs. Training data are optimised by considering image properties defined by metrics. The second module utilises adaptive instance normalisation and style embedding to enhance the quality of images from Module 1. Additional perceptual and patch-based loss functions are employed in training both modules. The performance of the new framework was compared with that of existing methods, (filtered backprojection (FBP) and ordered subset expectation maximisation (OSEM) without and with point spread function (OSEM-PSF)) with respect to correction for attenuation, patient motion and noise in simulated, NEMA phantom and human imaging data. Evaluation metrics included structural similarity (SSIM), peak-signal-to-noise ratio (PSNR), relative root mean squared error (rRMSE) for simulated data, and contrast-to-noise ratio (CNR) for NEMA phantom and human data.

Results

For simulated test data, the performance of the proposed framework was both qualitatively and quantitatively superior to that of FBP and OSEM. In the presence of noise, Module 1 generated images with a SSIM of 0.48 and higher. These images exhibited coarse structures that were subsequently refined by Module 2, yielding images with an SSIM higher than 0.71 (at least 22% higher than OSEM). The proposed method was robust against noise and motion. For NEMA phantoms, it achieved higher CNR values than OSEM. For human images, the CNR in brain regions was significantly higher than that of FBP and OSEM (p < 0.05, paired t-test). The CNR of images reconstructed with OSEM-PSF was similar to those reconstructed using the proposed method.

Conclusion

The proposed image reconstruction method can produce PET images with artefact correction.

Deep learning applications for quantitative and qualitative PET in PET/MR: technical and clinical unmet needs

We aim to provide an overview of technical and clinical unmet needs in deep learning (DL) applications for quantitative and qualitative PET in PET/MR, with a focus on attenuation correction, image enhancement, motion correction, kinetic modeling, and simulated data generation. (1) DL-based attenuation correction (DLAC) remains an area of limited exploration for pediatric whole-body PET/MR and lung-specific DLAC due to data shortages and technical limitations. (2) DL-based image enhancement approximating MR-guided regularized reconstruction with a high-resolution MR prior has shown promise in enhancing PET image quality. However, its clinical value has not been thoroughly evaluated across various radiotracers, and applications outside the head may pose challenges due to motion artifacts. (3) Robust training for DL-based motion correction requires pairs of motion-corrupted and motion-corrected PET/MR data. However, these pairs are rare. (4) DL-based approaches can address the limitations of dynamic PET, such as long scan durations that may cause patient discomfort and motion, providing new research opportunities. (5) Monte-Carlo simulations using anthropomorphic digital phantoms can provide extensive datasets to address the shortage of clinical data. This summary of technical/clinical challenges and potential solutions may provide research opportunities for the research community towards the clinical translation of DL solutions.

Linear diffusion noise boosted deep image prior for unsupervised sparse-view CT reconstruction

Objective.Deep learning has markedly enhanced the performance of sparse-view computed tomography reconstruction. However, the dependence of these methods on supervised training using high-quality paired datasets, and the necessity for retraining under varied physical acquisition conditions, constrain their generalizability across new imaging contexts and settings.Approach.To overcome these limitations, we propose an unsupervised approach grounded in the deep image prior framework. Our approach advances beyond the conventional single noise level input by incorporating multi-level linear diffusion noise, significantly mitigating the risk of overfitting. Furthermore, we embed non-local self-similarity as a deep implicit prior within a self-attention network structure, improving the model's capability to identify and utilize repetitive patterns throughout the image. Additionally, leveraging imaging physics, gradient backpropagation is performed between the image domain and projection data space to optimize network weights.Main Results.Evaluations with both simulated and clinical cases demonstrate our method's effective zero-shot adaptability across various projection views, highlighting its robustness and flexibility. Additionally, our approach effectively eliminates noise and streak artifacts while significantly restoring intricate image details.Significance. Our method aims to overcome the limitations in current supervised deep learning-based sparse-view CT reconstruction, offering improved generalizability and adaptability without the need for extensive paired training data.

High-resolution extracellular pH imaging of liver cancer with multiparametric MR using Deep Image Prior

Noninvasive extracellular pH (pHe) mapping with Biosensor Imaging of Redundant Deviation in Shifts (BIRDS) using MR spectroscopic imaging (MRSI) has been demonstrated on 3T clinical MR scanners at 8 × 8 × 10  mm3 spatial resolution and applied to study various liver cancer treatments. Although pHe imaging at higher resolution can be achieved by extending the acquisition time, a postprocessing method to increase the resolution is preferable, to minimize the duration spent by the subject in the MR scanner. In this work, we propose to improve the spatial resolution of pHe mapping with BIRDS by incorporating anatomical information in the form of multiparametric MRI and using an unsupervised deep-learning technique, Deep Image Prior (DIP). Specifically, we used high-resolution T 1 , T 2 , and diffusion-weighted imaging (DWI) MR images of rabbits with VX2 liver tumors as inputs to a U-Net architecture to provide anatomical information. U-Net parameters were optimized to minimize the difference between the output super-resolution image and the experimentally acquired low-resolution pHe image using the mean-absolute error. In this way, the super-resolution pHe image would be consistent with both anatomical MR images and the low-resolution pHe measurement from the scanner. The method was developed based on data from 49 rabbits implanted with VX2 liver tumors. For evaluation, we also acquired high-resolution pHe images from two rabbits, which were used as ground truth. The results indicate a good match between the spatial characteristics of the super-resolution images and the high-resolution ground truth, supported by the low pixelwise absolute error.

Single-Subject Deep-Learning Image Reconstruction With a Neural Optimization Transfer Algorithm for PET-Enabled Dual-Energy CT Imaging

Combining dual-energy computed tomography (DECT) with positron emission tomography (PET) offers many potential clinical applications but typically requires expensive hardware upgrades or increases radiation doses on PET/CT scanners due to an extra X-ray CT scan. The recent PET-enabled DECT method allows DECT imaging on PET/CT without requiring a second X-ray CT scan. It combines the already existing X-ray CT image with a 511 keV γ -ray CT (gCT) image reconstructed from time-of-flight PET emission data. A kernelized framework has been developed for reconstructing gCT image but this method has not fully exploited the potential of prior knowledge. Use of deep neural networks may explore the power of deep learning in this application. However, common approaches require a large database for training, which is impractical for a new imaging method like PET-enabled DECT. Here, we propose a single-subject method by using neural-network representation as a deep coefficient prior to improving gCT image reconstruction without population-based pre-training. The resulting optimization problem becomes the tomographic estimation of nonlinear neural-network parameters from gCT projection data. This complicated problem can be efficiently solved by utilizing the optimization transfer strategy with quadratic surrogates. Each iteration of the proposed neural optimization transfer algorithm includes: PET activity image update; gCT image update; and least-square neural-network learning in the gCT image domain. This algorithm is guaranteed to monotonically increase the data likelihood. Results from computer simulation, real phantom data and real patient data have demonstrated that the proposed method can significantly improve gCT image quality and consequent multi-material decomposition as compared to other methods.

Population-based deep image prior for dynamic PET denoising: A data-driven approach to improve parametric quantification

The high noise level of dynamic Positron Emission Tomography (PET) images degrades the quality of parametric images. In this study, we aim to improve the quality and quantitative accuracy of Ki images by utilizing deep learning techniques to reduce the noise in dynamic PET images. We propose a novel denoising technique, Population-based Deep Image Prior (PDIP), which integrates population-based prior information into the optimization process of Deep Image Prior (DIP). Specifically, the population-based prior image is generated from a supervised denoising model that is trained on a prompts-matched static PET dataset comprising 100 clinical studies. The 3D U-Net architecture is employed for both the supervised model and the following DIP optimization process. We evaluated the efficacy of PDIP for noise reduction in 25%-count and 100%-count dynamic PET images from 23 patients by comparing with two other baseline techniques: the Prompts-matched Supervised model (PS) and a conditional DIP (CDIP) model that employs the mean static PET image as the prior. Both the PS and CDIP models show effective noise reduction but result in smoothing and removal of small lesions. In addition, the utilization of a single static image as the prior in the CDIP model also introduces a similar tracer distribution to the denoised dynamic frames, leading to lower Ki in general as well as incorrect Ki in the descending aorta. By contrast, as the proposed PDIP model utilizes intrinsic image features from the dynamic dataset and a large clinical static dataset, it not only achieves comparable noise reduction as the supervised and CDIP models but also improves lesion Ki predictions.

Unsupervised motion artifact correction of turbo spin-echo MRI using deep image prior

PURPOSE

In MRI, motion artifacts can significantly degrade image quality. Motion artifact correction methods using deep neural networks usually required extensive training on large datasets, making them time-consuming and resource-intensive. In this paper, an unsupervised deep learning-based motion artifact correction method for turbo-spin echo MRI is proposed using the deep image prior framework.

THEORY AND METHODS

The proposed approach takes advantage of the high impedance to motion artifacts offered by the neural network parameterization to remove motion artifacts in MR images. The framework consists of parameterization of MR image, automatic spatial transformation, and motion simulation model. The proposed method synthesizes motion-corrupted images from the motion-corrected images generated by the convolutional neural network, where an optimization process minimizes the objective function between the synthesized images and the acquired images.

RESULTS

In the simulation study of 280 slices from 14 subjects, the proposed method showed a significant increase in the averaged structural similarity index measure by 0.2737 in individual coil images and by 0.4550 in the root-sum-of-square images. In addition, the ablation study demonstrated the effectiveness of each proposed component in correcting motion artifacts compared to the corrected images produced by the baseline method. The experiments on real motion dataset has shown its clinical potential.

CONCLUSION

The proposed method exhibited significant quantitative and qualitative improvements in correcting rigid and in-plane motion artifacts in MR images acquired using turbo spin-echo sequence.

Low-dose CT reconstruction using dataset-free learning

Low-Dose computer tomography (LDCT) is an ideal alternative to reduce radiation risk in clinical applications. Although supervised-deep-learning-based reconstruction methods have demonstrated superior performance compared to conventional model-driven reconstruction algorithms, they require collecting massive pairs of low-dose and norm-dose CT images for neural network training, which limits their practical application in LDCT imaging. In this paper, we propose an unsupervised and training data-free learning reconstruction method for LDCT imaging that avoids the requirement for training data. The proposed method is a post-processing technique that aims to enhance the initial low-quality reconstruction results, and it reconstructs the high-quality images by neural work training that minimizes the ℓ1-norm distance between the CT measurements and their corresponding simulated sinogram data, as well as the total variation (TV) value of the reconstructed image. Moreover, the proposed method does not require to set the weights for both the data fidelity term and the plenty term. Experimental results on the AAPM challenge data and LoDoPab-CT data demonstrate that the proposed method is able to effectively suppress the noise and preserve the tiny structures. Also, these results demonstrate the rapid convergence and low computational cost of the proposed method. The source code is available at https://github.com/linfengyu77/IRLDCT.

Spach Transformer: Spatial and Channel-Wise Transformer Based on Local and Global Self-Attentions for PET Image Denoising

Position emission tomography (PET) is widely used in clinics and research due to its quantitative merits and high sensitivity, but suffers from low signal-to-noise ratio (SNR). Recently convolutional neural networks (CNNs) have been widely used to improve PET image quality. Though successful and efficient in local feature extraction, CNN cannot capture long-range dependencies well due to its limited receptive field. Global multi-head self-attention (MSA) is a popular approach to capture long-range information. However, the calculation of global MSA for 3D images has high computational costs. In this work, we proposed an efficient spatial and channel-wise encoder-decoder transformer, Spach Transformer, that can leverage spatial and channel information based on local and global MSAs. Experiments based on datasets of different PET tracers, i.e., 18F-FDG, 18F-ACBC, 18F-DCFPyL, and 68Ga-DOTATATE, were conducted to evaluate the proposed framework. Quantitative results show that the proposed Spach Transformer framework outperforms state-of-the-art deep learning architectures.

Anatomically Guided PET Image Reconstruction Using Conditional Weakly-Supervised Multi-Task Learning Integrating Self-Attention

To address the lack of high-quality training labels in positron emission tomography (PET) imaging, weakly-supervised reconstruction methods that generate network-based mappings between prior images and noisy targets have been developed. However, the learned model has an intrinsic variance proportional to the average variance of the target image. To suppress noise and improve the accuracy and generalizability of the learned model, we propose a conditional weakly-supervised multi-task learning (MTL) strategy, in which an auxiliary task is introduced serving as an anatomical regularizer for the PET reconstruction main task. In the proposed MTL approach, we devise a novel multi-channel self-attention (MCSA) module that helps learn an optimal combination of shared and task-specific features by capturing both local and global channel-spatial dependencies. The proposed reconstruction method was evaluated on NEMA phantom PET datasets acquired at different positions in a PET/CT scanner and 26 clinical whole-body PET datasets. The phantom results demonstrate that our method outperforms state-of-the-art learning-free and weakly-supervised approaches obtaining the best noise/contrast tradeoff with a significant noise reduction of approximately 50.0% relative to the maximum likelihood (ML) reconstruction. The patient study results demonstrate that our method achieves the largest noise reductions of 67.3% and 35.5% in the liver and lung, respectively, as well as consistently small biases in 8 tumors with various volumes and intensities. In addition, network visualization reveals that adding the auxiliary task introduces more anatomical information into PET reconstruction than adding only the anatomical loss, and the developed MCSA can abstract features and retain PET image details.

Triplet-constrained deep hashing for chest X-ray image retrieval in COVID-19 assessment

Radiology images of the chest, such as computer tomography scans and X-rays, have been prominently used in computer-aided COVID-19 analysis. Learning-based radiology image retrieval has attracted increasing attention recently, which generally involves image feature extraction and finding matches in extensive image databases based on query images. Many deep hashing methods have been developed for chest radiology image search due to the high efficiency of retrieval using hash codes. However, they often overlook the complex triple associations between images; that is, images belonging to the same category tend to share similar characteristics and vice versa. To this end, we develop a triplet-constrained deep hashing (TCDH) framework for chest radiology image retrieval to facilitate automated analysis of COVID-19. The TCDH consists of two phases, including (a) feature extraction and (b) image retrieval. For feature extraction, we have introduced a triplet constraint and an image reconstruction task to enhance discriminative ability of learned features, and these features are then converted into binary hash codes to capture semantic information. Specifically, the triplet constraint is designed to pull closer samples within the same category and push apart samples from different categories. Additionally, an auxiliary image reconstruction task is employed during feature extraction to help effectively capture anatomical structures of images. For image retrieval, we utilize learned hash codes to conduct searches for medical images. Extensive experiments on 30,386 chest X-ray images demonstrate the superiority of the proposed method over several state-of-the-art approaches in automated image search. The code is now available online.

Efficient image reconstruction for a small animal PET system with dual-layer-offset detector design

BACKGROUND

A compact PET/SPECT/CT system Inliview-3000B has been developed to provide multi-modality information on small animals for biomedical research. Its PET subsystem employed a dual-layer-offset detector design for depth-of-interaction capability and higher detection efficiency, but the irregular design caused some difficulties in calculating the normalization factors and the sensitivity map. Besides, the relatively larger (2 mm) crystal cross-section size also posed a challenge to high-resolution image reconstruction.

PURPOSE

We present an efficient image reconstruction method to achieve high imaging performance for the PET subsystem of Inliview-3000B.

METHODS

List mode reconstruction with efficient system modeling was used for the PET imaging. We adopt an on-the-fly multi-ray tracing method with random crystal sampling to model the solid angle, crystal penetration and object attenuation effect, and modify the system response model during each iteration to improve the reconstruction performance and computational efficiency. We estimate crystal efficiency with a novel iterative approach that combines measured cylinder phantom data with simulated line-of-response (LOR)-based factors for normalization correction before reconstruction. Since it is necessary to calculate normalization factors and the sensitivity map, we stack the two crystal layers together and extend the conventional data organization method here to index all useful LORs. Simulations and experiments were performed to demonstrate the feasibility and advantage of the proposed method.

RESULTS

Simulation results showed that the iterative algorithm for crystal efficiency estimation could achieve good accuracy. NEMA image quality phantom studies have demonstrated the superiority of random sampling, which is able to achieve good imaging performance with much less computation than traditional uniform sampling. In the spatial resolution evaluation based on the mini-Derenzo phantom, 1.1 mm hot rods could be identified with the proposed reconstruction method. Reconstruction of double mice and a rat showed good spatial resolution and a high signal-to-noise ratio, and organs with higher uptake could be recognized well.

CONCLUSION

The results validated the superiority of introducing randomness into reconstruction, and demonstrated its reliability for high-performance imaging. The Inliview-3000B PET subsystem with the proposed image reconstruction can provide rich and detailed information on small animals for preclinical research.

AIRPORT: A Data Consistency Constrained Deep Temporal Extrapolation Method To Improve Temporal Resolution In Contrast Enhanced CT Imaging

Typical tomographic image reconstruction methods require that the imaged object is static and stationary during the time window to acquire a minimally complete data set. The violation of this requirement leads to temporal-averaging errors in the reconstructed images. For a fixed gantry rotation speed, to reduce the errors, it is desired to reconstruct images using data acquired over a narrower angular range, i.e., with a higher temporal resolution. However, image reconstruction with a narrower angular range violates the data sufficiency condition, resulting in severe data-insufficiency-induced errors. The purpose of this work is to decouple the trade-off between these two types of errors in contrast-enhanced computed tomography (CT) imaging. We demonstrated that using the developed data consistency constrained deep temporal extrapolation method (AIRPORT), the entire time-varying imaged object can be accurately reconstructed with 40 frames-per-second temporal resolution, the time window needed to acquire a single projection view data using a typical C-arm cone-beam CT system. AIRPORT is applicable to general non-sparse imaging tasks using a single short-scan data acquisition.

Deep learning-based PET image denoising and reconstruction: a review

This review focuses on positron emission tomography (PET) imaging algorithms and traces the evolution of PET image reconstruction methods. First, we provide an overview of conventional PET image reconstruction methods from filtered backprojection through to recent iterative PET image reconstruction algorithms, and then review deep learning methods for PET data up to the latest innovations within three main categories. The first category involves post-processing methods for PET image denoising. The second category comprises direct image reconstruction methods that learn mappings from sinograms to the reconstructed images in an end-to-end manner. The third category comprises iterative reconstruction methods that combine conventional iterative image reconstruction with neural-network enhancement. We discuss future perspectives on PET imaging and deep learning technology.

An iterative reconstruction algorithm for unsupervised PET image

Objective.In recent years, convolutional neural networks (CNNs) have shown great potential in positron emission tomography (PET) image reconstruction. However, most of them rely on many low-quality and high-quality reference PET image pairs for training, which are not always feasible in clinical practice. On the other hand, many works improve the quality of PET image reconstruction by adding explicit regularization or optimizing the network structure, which may lead to complex optimization problems.Approach.In this paper, we develop a novel iterative reconstruction algorithm by integrating the deep image prior (DIP) framework, which only needs the prior information (e.g. MRI) and sinogram data of patients. To be specific, we construct the objective function as a constrained optimization problem and utilize the existing PET image reconstruction packages to streamline calculations. Moreover, to further improve both the reconstruction quality and speed, we introduce the Nesterov's acceleration part and the restart mechanism in each iteration.Main results.2D experiments on PET data sets based on computer simulations and real patients demonstrate that our proposed algorithm can outperform existing MLEM-GF, KEM and DIPRecon methods.Significance.Unlike traditional CNN methods, the proposed algorithm does not rely on large data sets, but only leverages inter-patient information. Furthermore, we enhance reconstruction performance by optimizing the iterative algorithm. Notably, the proposed method does not require much modification of the basic algorithm, allowing for easy integration into standard implementations.

M3S-Net: multi-modality multi-branch multi-self-attention network with structure-promoting loss for low-dose PET/CT enhancement

Objective.PET (Positron Emission Tomography) inherently involves radiotracer injections and long scanning time, which raises concerns about the risk of radiation exposure and patient comfort. Reductions in radiotracer dosage and acquisition time can lower the potential risk and improve patient comfort, respectively, but both will also reduce photon counts and hence degrade the image quality. Therefore, it is of interest to improve the quality of low-dose PET images.Approach.A supervised multi-modality deep learning model, named M3S-Net, was proposed to generate standard-dose PET images (60 s per bed position) from low-dose ones (10 s per bed position) and the corresponding CT images. Specifically, we designed a multi-branch convolutional neural network with multi-self-attention mechanisms, which first extracted features from PET and CT images in two separate branches and then fused the features to generate the final generated PET images. Moreover, a novel multi-modality structure-promoting term was proposed in the loss function to learn the anatomical information contained in CT images.Main results.We conducted extensive numerical experiments on real clinical data collected from local hospitals. Compared with state-of-the-art methods, the proposed M3S-Net not only achieved higher objective metrics and better generated tumors, but also performed better in preserving edges and suppressing noise and artifacts.Significance.The experimental results of quantitative metrics and qualitative displays demonstrate that the proposed M3S-Net can generate high-quality PET images from low-dose ones, which are competable to standard-dose PET images. This is valuable in reducing PET acquisition time and has potential applications in dynamic PET imaging.

A Review of Hypoxia Imaging Using 18F-Fluoromisonidazole Positron Emission Tomography

Tumor hypoxia is an essential factor related to malignancy, prognosis, and resistance to treatment. Positron emission tomography (PET) is a modality that visualizes the distribution of radiopharmaceuticals administered into the body. PET imaging with [18F]fluoromisonidazole ([18F]FMISO) identifies hypoxic tissues. Unlike [18F]fluorodeoxyglucose ([18F]FDG)-PET, fasting is not necessary for [18F]FMISO-PET, but the waiting time from injection to image acquisition needs to be relatively long (e.g., 2-4 h). [18F]FMISO-PET images can be displayed on an ordinary commercial viewer on a personal computer (PC). While visual assessment is fundamental, various quantitative indices such as tumor-to-muscle ratio have also been proposed. Several novel hypoxia tracers have been invented to compensate for the limitations of [18F]FMISO.

Recent advances in combined Positron Emission Tomography and Magnetic Resonance Imaging

Hybrid imaging modalities combine two or more medical imaging techniques offering exciting new possibilities to image the structure, function and biochemistry of the human body in far greater detail than has previously been possible to improve patient diagnosis. In this context, simultaneous Positron Emission Tomography and Magnetic Resonance (PET/MR) imaging offers great complementary information, but it also poses challenges from the point of view of hardware and software compatibility. The PET signal may interfere with the MR magnetic field and vice-versa, posing several challenges and constrains in the PET instrumentation for PET/MR systems. Additionally, anatomical maps are needed to properly apply attenuation and scatter corrections to the resulting reconstructed PET images, as well motion estimates to minimize the effects of movement throughout the acquisition. In this review, we summarize the instrumentation implemented in modern PET scanners to overcome these limitations, describing the historical development of hybrid PET/MR scanners. We pay special attention to the methods used in PET to achieve attenuation, scatter and motion correction when it is combined with MR, and how both imaging modalities may be combined in PET image reconstruction algorithms.

Deep Generalized Learning Model for PET Image Reconstruction

Low-count positron emission tomography (PET) imaging is challenging because of the ill-posedness of this inverse problem. Previous studies have demonstrated that deep learning (DL) holds promise for achieving improved low-count PET image quality. However, almost all data-driven DL methods suffer from fine structure degradation and blurring effects after denoising. Incorporating DL into the traditional iterative optimization model can effectively improve its image quality and recover fine structures, but little research has considered the full relaxation of the model, resulting in the performance of this hybrid model not being sufficiently exploited. In this paper, we propose a learning framework that deeply integrates DL and an alternating direction of multipliers method (ADMM)-based iterative optimization model. The innovative feature of this method is that we break the inherent forms of the fidelity operators and use neural networks to process them. The regularization term is deeply generalized. The proposed method is evaluated on simulated data and real data. Both the qualitative and quantitative results show that our proposed neural network method can outperform partial operator expansion-based neural network methods, neural network denoising methods and traditional methods.

A generalization of the maximum likelihood expectation maximization (MLEM) method: Masked-MLEM

Objective.In our previous work on image reconstruction for single-layer collimatorless scintigraphy, we developed the min-min weighted robust least squares (WRLS) optimization algorithm to address the challenge of reconstructing images when both the system matrix and the projection data are uncertain. Whereas the WRLS algorithm has been successful in two-dimensional (2D) reconstruction, expanding it to three-dimensional (3D) reconstruction is difficult since the WRLS optimization problem is neither smooth nor strongly-convex. To overcome these difficulties and achieve robust image reconstruction in the presence of system uncertainties and projection noise, we propose a generalized iterative method based on the maximum likelihood expectation maximization (MLEM) algorithm, hereinafter referred to as the Masked-MLEM algorithm.Approach.In the Masked-MLEM algorithm, only selected subsets ('masks') from the system matrix and the projection contribute to the image update to satisfy the constraints imposed by the system uncertainties. We validate the Masked-MLEM algorithm and compare it to the standard MLEM algorithm using experimental data obtained from both collimated and uncollimated imaging instruments, including parallel-hole collimated SPECT, 2D collimatorless scintigraphy, and 3D collimatorless tomography. Additionally, we conduct comprehensive Monte Carlo simulations for 3D collimatorless tomography to further validate the effectiveness of the Masked-MLEM algorithm in handling different levels of system uncertainties.Main results.The Masked-MLEM and standard MLEM reconstructions are similar in cases with negligible system uncertainties, whereas the Masked-MLEM algorithm outperforms the standard MLEM algorithm when the system matrix is an approximation. Importantly, the Masked-MLEM algorithm ensures reliable image reconstruction across varying levels of system uncertainties.Significance.With a good choice of system uncertainty and without requiring accurate knowledge of the actual system matrix, the Masked-MLEM algorithm yields more robust image reconstruction than the standard MLEM algorithm, effectively reducing the likelihood of erroneously reconstructing higher activities in regions without radioactive sources.

Artificial Intelligence and Deep Learning for Advancing PET Image Reconstruction: State-of-the-Art and Future Directions

Positron emission tomography (PET) is vital for diagnosing diseases and monitoring treatments. Conventional image reconstruction (IR) techniques like filtered backprojection and iterative algorithms are powerful but face limitations. PET IR can be seen as an image-to-image translation. Artificial intelligence (AI) and deep learning (DL) using multilayer neural networks enable a new approach to this computer vision task. This review aims to provide mutual understanding for nuclear medicine professionals and AI researchers. We outline fundamentals of PET imaging as well as state-of-the-art in AI-based PET IR with its typical algorithms and DL architectures. Advances improve resolution and contrast recovery, reduce noise, and remove artifacts via inferred attenuation and scatter correction, sinogram inpainting, denoising, and super-resolution refinement. Kernel-priors support list-mode reconstruction, motion correction, and parametric imaging. Hybrid approaches combine AI with conventional IR. Challenges of AI-assisted PET IR include availability of training data, cross-scanner compatibility, and the risk of hallucinated lesions. The need for rigorous evaluations, including quantitative phantom validation and visual comparison of diagnostic accuracy against conventional IR, is highlighted along with regulatory issues. First approved AI-based applications are clinically available, and its impact is foreseeable. Emerging trends, such as the integration of multimodal imaging and the use of data from previous imaging visits, highlight future potentials. Continued collaborative research promises significant improvements in image quality, quantitative accuracy, and diagnostic performance, ultimately leading to the integration of AI-based IR into routine PET imaging protocols.

Denoising and uncertainty estimation in parameter mapping with approximate Bayesian deep image priors

PURPOSE

To mitigate the problem of noisy parameter maps with high uncertainties by casting parameter mapping as a denoising task based on Deep Image Priors.

METHODS

We extend the concept of denoising with Deep Image Prior (DIP) into parameter mapping by treating the output of an image-generating network as a parametrization of tissue parameter maps. The method implicitly denoises the parameter mapping process by filtering low-level image features with an untrained convolutional neural network (CNN). Our implementation includes uncertainty estimation from Bernoulli approximate variational inference, implemented with MC dropout, which provides model uncertainty in each voxel of the denoised parameter maps. The method is modular, so the specifics of different applications (e.g., T1 mapping) separate into application-specific signal equation blocks. We evaluate the method on variable flip angle T1 mapping, multi-echo T2 mapping, and apparent diffusion coefficient mapping.

RESULTS

We found that deep image prior adapts successfully to several applications in parameter mapping. In all evaluations, the method produces noise-reduced parameter maps with decreased uncertainty compared to conventional methods. The downsides of the proposed method are the long computational time and the introduction of some bias from the denoising prior.

CONCLUSION

DIP successfully denoise the parameter mapping process and applies to several applications with limited hyperparameter tuning. Further, it is easy to implement since DIP methods do not use network training data. Although time-consuming, uncertainty information from MC dropout makes the method more robust and provides useful information when properly calibrated.

Image reconstruction using UNET-transformer network for fast and low-dose PET scans

INTRODUCTION

Low-dose and fast PET imaging (low-count PET) play a significant role in enhancing patient safety, healthcare efficiency, and patient comfort during medical imaging procedures. To achieve high-quality images with low-count PET scans, effective reconstruction models are crucial for denoising and enhancing image quality. The main goal of this paper is to develop an effective and accurate deep learning-based method for reconstructing low-count PET images, which is a challenging problem due to the limited amount of available data and the high level of noise in the acquired images. The proposed method aims to improve the quality of reconstructed PET images while preserving important features, such as edges and small details, by combining the strengths of UNET and Transformer networks.

MATERIAL AND METHODS

The proposed TrUNET-MAPEM model integrates a residual UNET-transformer regularizer into the unrolled maximum a posteriori expectation maximization (MAPEM) algorithm for PET image reconstruction. A loss function based on a combination of structural similarity index (SSIM) and mean squared error (MSE) is utilized to evaluate the accuracy of the reconstructed images. The simulated dataset was generated using the Brainweb phantom, while the real patient dataset was acquired using a Siemens Biograph mMR PET scanner. We also implemented state-of-the-art methods for comparison purposes: OSEM, MAPOSEM, and supervised learning using 3D-UNET network. The reconstructed images are compared to ground truth images using metrics such as peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), and relative root mean square error (rRMSE) to quantitatively evaluate the accuracy of the reconstructed images.

RESULTS

Our proposed TrUNET-MAPEM approach was evaluated using both simulated and real patient data. For the patient data, our model achieved an average PSNR of 33.72 dB, an average SSIM of 0.955, and an average rRMSE of 0.39. These results outperformed other methods which had average PSNRs of 36.89 dB, 34.12 dB, and 33.52 db, average SSIMs of 0.944, 0.947, and 0.951, and average rRMSEs of 0.59, 0.49, and 0.42. For the simulated data, our model achieved an average PSNR of 31.23 dB, an average SSIM of 0.95, and an average rRMSE of 0.55. These results also outperformed other state-of-the-art methods, such as OSEM, MAPOSEM, and 3DUNET-MAPEM. The model demonstrates the potential for clinical use by successfully reconstructing smooth images while preserving edges. The comparison with other methods demonstrates the superiority of our approach, as it outperforms all other methods for all three metrics.

CONCLUSION

The proposed TrUNET-MAPEM model presents a significant advancement in the field of low-count PET image reconstruction. The results demonstrate the potential for clinical use, as the model can produce images with reduced noise levels and better edge preservation compared to other reconstruction and post-processing algorithms. The proposed approach may have important clinical applications in the early detection and diagnosis of various diseases.

Longitudinal amyloid and tau PET imaging in Alzheimer's disease: A systematic review of methodologies and factors affecting quantification

Deposition of amyloid and tau pathology can be quantified in vivo using positron emission tomography (PET). Accurate longitudinal measurements of accumulation from these images are critical for characterizing the start and spread of the disease. However, these measurements are challenging; precision and accuracy can be affected substantially by various sources of errors and variability. This review, supported by a systematic search of the literature, summarizes the current design and methodologies of longitudinal PET studies. Intrinsic, biological causes of variability of the Alzheimer's disease (AD) protein load over time are then detailed. Technical factors contributing to longitudinal PET measurement uncertainty are highlighted, followed by suggestions for mitigating these factors, including possible techniques that leverage shared information between serial scans. Controlling for intrinsic variability and reducing measurement uncertainty in longitudinal PET pipelines will provide more accurate and precise markers of disease evolution, improve clinical trial design, and aid therapy response monitoring.

Deep learning study on the mechanism of edge artifacts in point spread function reconstruction for numerical brain images

OBJECTIVE

Non-blinded image deblurring with deep learning was performed on blurred numerical brain images without point spread function (PSF) reconstruction to obtain edge artifacts (EA)-free images. This study uses numerical simulation to investigate the mechanism of EA in PSF reconstruction based on the spatial frequency characteristics of EA-free images.

METHODS

In 256 × 256 matrix brain images, the signal values of gray matter (GM), white matter, and cerebrospinal fluid were set to 1, 0.25, and 0.05, respectively. We assumed ideal projection data of a two-dimensional (2D) parallel beam with no degradation factors other than detector response blur to precisely grasp EA using the PSF reconstruction algorithm from blurred projection data. The detector response was assumed to be a shift-invariant and one-dimensional (1D) Gaussian function with 2-5 mm full width at half maximum (FWHM). Images without PSF reconstruction (non-PSF), PSF reconstruction without regularization (PSF) and with regularization of relative difference function (PSF-RD) were generated by ordered subset expectation maximization (OSEM). For non-PSF, the image deblurring with a deep image prior (DIP) was applied using a 2D Gaussian function with 2-5 mm FWHM. The 1D object-specific modulation transfer function (1D-OMTF), which is the ratio of 1D amplitude spectrum of the original and reconstructed images, was used as the index of spatial frequency characteristics.

RESULTS

When the detector response was greater than 3 mm FWHM, EA in PSF was observed in GM borders and narrow GM. No remarkable EA was observed in the DIP, and the FWHM estimated from the recovery coefficient for the deblurred image of non-PSF at 5 mm FWHM was reduced to 3 mm or less. PSF of 5 mm FWHM showed higher spatial frequency characteristics than that of DIP up to around 2.2 cycles/cm but was lower than the latter after 3 cycles/cm. PSF-RD showed almost the same spatial frequency characteristics as that of DIP above 3 cycles/cm but was inferior below 3 cycles/cm. PSF-RD has a lower spatial resolution than DIP.

CONCLUSIONS

Unlike DIP, PSF lacks high-frequency components around the Nyquist frequency, generating EA. PSF-RD mitigates EA while simultaneously suppressing the signal, diminishing spatial resolution.

Super-resolution biomedical imaging via reference-free statistical implicit neural representation

Objective.Supervised deep learning for image super-resolution (SR) has limitations in biomedical imaging due to the lack of large amounts of low- and high-resolution image pairs for model training. In this work, we propose a reference-free statistical implicit neural representation (INR) framework, which needs only a single or a few observed low-resolution (LR) image(s), to generate high-quality SR images.Approach.The framework models the statistics of the observed LR images via maximum likelihood estimation and trains the INR network to represent the latent high-resolution (HR) image as a continuous function in the spatial domain. The INR network is constructed as a coordinate-based multi-layer perceptron, whose inputs are image spatial coordinates and outputs are corresponding pixel intensities. The trained INR not only constrains functional smoothness but also allows an arbitrary scale in SR imaging.Main results.We demonstrate the efficacy of the proposed framework on various biomedical images, including computed tomography (CT), magnetic resonance imaging (MRI), fluorescence microscopy, and ultrasound images, across different SR magnification scales of 2×, 4×, and 8×. A limited number of LR images were used for each of the SR imaging tasks to show the potential of the proposed statistical INR framework.Significance.The proposed method provides an urgently needed unsupervised deep learning framework for numerous biomedical SR applications that lack HR reference images.

AI for PET image reconstruction

Image reconstruction for positron emission tomography (PET) has been developed over many decades, with advances coming from improved modelling of the data statistics and improved modelling of the imaging physics. However, high noise and limited spatial resolution have remained issues in PET imaging, and state-of-the-art PET reconstruction has started to exploit other medical imaging modalities (such as MRI) to assist in noise reduction and enhancement of PET's spatial resolution. Nonetheless, there is an ongoing drive towards not only improving image quality, but also reducing the injected radiation dose and reducing scanning times. While the arrival of new PET scanners (such as total body PET) is helping, there is always a need to improve reconstructed image quality due to the time and count limited imaging conditions. Artificial intelligence (AI) methods are now at the frontier of research for PET image reconstruction. While AI can learn the imaging physics as well as the noise in the data (when given sufficient examples), one of the most common uses of AI arises from exploiting databases of high-quality reference examples, to provide advanced noise compensation and resolution recovery. There are three main AI reconstruction approaches: (i) direct data-driven AI methods which rely on supervised learning from reference data, (ii) iterative (unrolled) methods which combine our physics and statistical models with AI learning from data, and (iii) methods which exploit AI with our known models, but crucially can offer benefits even in the absence of any example training data whatsoever. This article reviews these methods, considering opportunities and challenges of AI for PET reconstruction.

Evaluation of MR anatomically-guided PET reconstruction using a convolutional neural network in PSMA patients

Background. Recently, approaches have utilized the superior anatomical information provided by magnetic resonance imaging (MRI) to guide the reconstruction of positron emission tomography (PET). One of those approaches is the Bowsher's prior, which has been accelerated lately with a convolutional neural network (CNN) to reconstruct MR-guided PET in the imaging domain in routine clinical imaging. Two differently trained Bowsher-CNN methods (B-CNN0 and B-CNN) have been trained and tested on brain PET/MR images with non-PSMA tracers, but so far, have not been evaluated in other anatomical regions yet.Methods. A NEMA phantom with five of its six spheres filled with the same, calibrated concentration of 18F-DCFPyL-PSMA, and thirty-two patients (mean age 64 ± 7 years) with biopsy-confirmed PCa were used in this study. Reconstruction with either of the two available Bowsher-CNN methods were performed on the conventional MR-based attenuation correction (MRAC) and T1-MR images in the imaging domain. Detectable volume of the spheres and tumors, relative contrast recovery (CR), and background variation (BV) were measured for the MRAC and the Bowsher-CNN images, and qualitative assessment was conducted by ranking the image sharpness and quality by two experienced readers.Results. For the phantom study, the B-CNN produced 12.7% better CR compared to conventional reconstruction. The small sphere volume (<1.8 ml) detectability improved from MRAC to B-CNN by nearly 13%, while measured activity was higher than the ground-truth by 8%. The signal-to-noise ratio, CR, and BV were significantly improved (p< 0.05) in B-CNN images of the tumor. The qualitative analysis determined that tumor sharpness was excellent in 76% of the PET images reconstructed with the B-CNN method, compared to conventional reconstruction.Conclusions. Applying the MR-guided B-CNN in clinical prostate PET/MR imaging improves some quantitative, as well as qualitative imaging measures. The measured improvements in the phantom are also clearly translated into clinical application.

A review of PET attenuation correction methods for PET-MR

Despite being thirteen years since the installation of the first PET-MR system, the scanners constitute a very small proportion of the total hybrid PET systems installed. This is in stark contrast to the rapid expansion of the PET-CT scanner, which quickly established its importance in patient diagnosis within a similar timeframe. One of the main hurdles is the development of an accurate, reproducible and easy-to-use method for attenuation correction. Quantitative discrepancies in PET images between the manufacturer-provided MR methods and the more established CT- or transmission-based attenuation correction methods have led the scientific community in a continuous effort to develop a robust and accurate alternative. These can be divided into four broad categories: (i) MR-based, (ii) emission-based, (iii) atlas-based and the (iv) machine learning-based attenuation correction, which is rapidly gaining momentum. The first is based on segmenting the MR images in various tissues and allocating a predefined attenuation coefficient for each tissue. Emission-based attenuation correction methods aim in utilising the PET emission data by simultaneously reconstructing the radioactivity distribution and the attenuation image. Atlas-based attenuation correction methods aim to predict a CT or transmission image given an MR image of a new patient, by using databases containing CT or transmission images from the general population. Finally, in machine learning methods, a model that could predict the required image given the acquired MR or non-attenuation-corrected PET image is developed by exploiting the underlying features of the images. Deep learning methods are the dominant approach in this category. Compared to the more traditional machine learning, which uses structured data for building a model, deep learning makes direct use of the acquired images to identify underlying features. This up-to-date review goes through the literature of attenuation correction approaches in PET-MR after categorising them. The various approaches in each category are described and discussed. After exploring each category separately, a general overview is given of the current status and potential future approaches along with a comparison of the four outlined categories.

Enhanced PET imaging using progressive conditional deep image prior

Objective.Unsupervised learning-based methods have been proven to be an effective way to improve the image quality of positron emission tomography (PET) images when a large dataset is not available. However, when the gap between the input image and the target PET image is large, direct unsupervised learning can be challenging and easily lead to reduced lesion detectability. We aim to develop a new unsupervised learning method to improve lesion detectability in patient studies.Approach.We applied the deep progressive learning strategy to bridge the gap between the input image and the target image. The one-step unsupervised learning is decomposed into two unsupervised learning steps. The input image of the first network is an anatomical image and the input image of the second network is a PET image with a low noise level. The output of the first network is also used as the prior image to generate the target image of the second network by iterative reconstruction method.Results.The performance of the proposed method was evaluated through the phantom and patient studies and compared with non-deep learning, supervised learning and unsupervised learning methods. The results showed that the proposed method was superior to non-deep learning and unsupervised methods, and was comparable to the supervised method.Significance.A progressive unsupervised learning method was proposed, which can improve image noise performance and lesion detectability.

Fully 3D implementation of the end-to-end deep image prior-based PET image reconstruction using block iterative algorithm

Objective. Deep image prior (DIP) has recently attracted attention owing to its unsupervised positron emission tomography (PET) image reconstruction method, which does not require any prior training dataset. In this paper, we present the first attempt to implement an end-to-end DIP-based fully 3D PET image reconstruction method that incorporates a forward-projection model into a loss function.Approach. A practical implementation of a fully 3D PET image reconstruction could not be performed at present because of a graphics processing unit memory limitation. Consequently, we modify the DIP optimization to a block iteration and sequential learning of an ordered sequence of block sinograms. Furthermore, the relative difference penalty (RDP) term is added to the loss function to enhance the quantitative accuracy of the PET image.Main results. We evaluated our proposed method using Monte Carlo simulation with [18F]FDG PET data of a human brain and a preclinical study on monkey-brain [18F]FDG PET data. The proposed method was compared with the maximum-likelihood expectation maximization (EM), maximuma posterioriEM with RDP, and hybrid DIP-based PET reconstruction methods. The simulation results showed that, compared with other algorithms, the proposed method improved the PET image quality by reducing statistical noise and better preserved the contrast of brain structures and inserted tumors. In the preclinical experiment, finer structures and better contrast recovery were obtained with the proposed method.Significance.The results indicated that the proposed method could produce high-quality images without a prior training dataset. Thus, the proposed method could be a key enabling technology for the straightforward and practical implementation of end-to-end DIP-based fully 3D PET image reconstruction.

Fast and low-dose medical imaging generation empowered by hybrid deep-learning and iterative reconstruction

Fast and low-dose reconstructions of medical images are highly desired in clinical routines. We propose a hybrid deep-learning and iterative reconstruction (hybrid DL-IR) framework and apply it for fast magnetic resonance imaging (MRI), fast positron emission tomography (PET), and low-dose computed tomography (CT) image generation tasks. First, in a retrospective MRI study (6,066 cases), we demonstrate its capability of handling 3- to 10-fold under-sampled MR data, enabling organ-level coverage with only 10- to 100-s scan time; second, a low-dose CT study (142 cases) shows that our framework can successfully alleviate the noise and streak artifacts in scans performed with only 10% radiation dose (0.61 mGy); and last, a fast whole-body PET study (131 cases) allows us to faithfully reconstruct tumor-induced lesions, including small ones (<4 mm), from 2- to 4-fold-accelerated PET acquisition (30-60 s/bp). This study offers a promising avenue for accurate and high-quality image reconstruction with broad clinical value.

K-UNN: k-space interpolation with untrained neural network

Recently, untrained neural networks (UNNs) have shown satisfactory performances for MR image reconstruction on random sampling trajectories without using additional full-sampled training data. However, the existing UNN-based approaches lack the modeling of physical priors, resulting in poor performance in some common scenarios (e.g., partial Fourier (PF), regular sampling, etc.) and the lack of theoretical guarantees for reconstruction accuracy. To bridge this gap, we propose a safeguarded k-space interpolation method for MRI using a specially designed UNN with a tripled architecture driven by three physical priors of the MR images (or k-space data), including transform sparsity, coil sensitivity smoothness, and phase smoothness. We also prove that the proposed method guarantees tight bounds for interpolated k-space data accuracy. Finally, ablation experiments show that the proposed method can characterize the physical priors of MR images well. Additionally, experiments show that the proposed method consistently outperforms traditional parallel imaging methods and existing UNNs, and is even competitive against supervised-trained deep learning methods in PF and regular undersampling reconstruction.

Untrained Neural Network Priors for Inverse Imaging Problems: A Survey

In recent years, advancements in machine learning (ML) techniques, in particular, deep learning (DL) methods have gained a lot of momentum in solving inverse imaging problems, often surpassing the performance provided by hand-crafted approaches. Traditionally, analytical methods have been used to solve inverse imaging problems such as image restoration, inpainting, and superresolution. Unlike analytical methods for which the problem is explicitly defined and the domain knowledge is carefully engineered into the solution, DL models do not benefit from such prior knowledge and instead make use of large datasets to predict an unknown solution to the inverse problem. Recently, a new paradigm of training deep models using a single image, named untrained neural network prior (UNNP) has been proposed to solve a variety of inverse tasks, e.g., restoration and inpainting. Since then, many researchers have proposed various applications and variants of UNNP. In this paper, we present a comprehensive review of such studies and various UNNP applications for different tasks and highlight various open research problems which require further research.

Low-count whole-body PET/MRI restoration: an evaluation of dose reduction spectrum and five state-of-the-art artificial intelligence models

PURPOSE

To provide a holistic and complete comparison of the five most advanced AI models in the augmentation of low-dose 18F-FDG PET data over the entire dose reduction spectrum.

METHODS

In this multicenter study, five AI models were investigated for restoring low-count whole-body PET/MRI, covering convolutional benchmarks - U-Net, enhanced deep super-resolution network (EDSR), generative adversarial network (GAN) - and the most cutting-edge image reconstruction transformer models in computer vision to date - Swin transformer image restoration network (SwinIR) and EDSR-ViT (vision transformer). The models were evaluated against six groups of count levels representing the simulated 75%, 50%, 25%, 12.5%, 6.25%, and 1% (extremely ultra-low-count) of the clinical standard 3 MBq/kg 18F-FDG dose. The comparisons were performed upon two independent cohorts - (1) a primary cohort from Stanford University and (2) a cross-continental external validation cohort from Tübingen University - in order to ensure the findings are generalizable. A total of 476 original count and simulated low-count whole-body PET/MRI scans were incorporated into this analysis.

RESULTS

For low-count PET restoration on the primary cohort, the mean structural similarity index (SSIM) scores for dose 6.25% were 0.898 (95% CI, 0.887-0.910) for EDSR, 0.893 (0.881-0.905) for EDSR-ViT, 0.873 (0.859-0.887) for GAN, 0.885 (0.873-0.898) for U-Net, and 0.910 (0.900-0.920) for SwinIR. In continuation, SwinIR and U-Net's performances were also discreetly evaluated at each simulated radiotracer dose levels. Using the primary Stanford cohort, the mean diagnostic image quality (DIQ; 5-point Likert scale) scores of SwinIR restoration were 5 (SD, 0) for dose 75%, 4.50 (0.535) for dose 50%, 3.75 (0.463) for dose 25%, 3.25 (0.463) for dose 12.5%, 4 (0.926) for dose 6.25%, and 2.5 (0.534) for dose 1%.

CONCLUSION

Compared to low-count PET images, with near-to or nondiagnostic images at higher dose reduction levels (up to 6.25%), both SwinIR and U-Net significantly improve the diagnostic quality of PET images. A radiotracer dose reduction to 1% of the current clinical standard radiotracer dose is out of scope for current AI techniques.

Synthetic PET via Domain Translation of 3-D MRI

Historically, patient datasets have been used to develop and validate various reconstruction algorithms for PET/MRI and PET/CT. To enable such algorithm development, without the need for acquiring hundreds of patient exams, in this article we demonstrate a deep learning technique to generate synthetic but realistic whole-body PET sinograms from abundantly available whole-body MRI. Specifically, we use a dataset of 56 18F-FDG-PET/MRI exams to train a 3-D residual UNet to predict physiologic PET uptake from whole-body T1-weighted MRI. In training, we implemented a balanced loss function to generate realistic uptake across a large dynamic range and computed losses along tomographic lines of response to mimic the PET acquisition. The predicted PET images are forward projected to produce synthetic PET (sPET) time-of-flight (ToF) sinograms that can be used with vendor-provided PET reconstruction algorithms, including using CT-based attenuation correction (CTAC) and MR-based attenuation correction (MRAC). The resulting synthetic data recapitulates physiologic 18F-FDG uptake, e.g., high uptake localized to the brain and bladder, as well as uptake in liver, kidneys, heart, and muscle. To simulate abnormalities with high uptake, we also insert synthetic lesions. We demonstrate that this sPET data can be used interchangeably with real PET data for the PET quantification task of comparing CTAC and MRAC methods, achieving ≤ 7.6% error in mean-SUV compared to using real data. These results together show that the proposed sPET data pipeline can be reasonably used for development, evaluation, and validation of PET/MRI reconstruction methods.

Anatomically guided reconstruction improves lesion quantitation and detectability in bone SPECT/CT

Bone single-photon emission computed tomography (SPECT)/computed tomography (CT) imaging suffers from poor spatial resolution, but the image quality can be improved during SPECT reconstruction by using anatomical information derived from CT imaging. The purpose of this work was to compare two different anatomically guided SPECT reconstruction methods to ordered subsets expectation maximization (OSEM) which is the most commonly used reconstruction method in nuclear medicine. The comparison was done in terms of lesion quantitation and lesion detectability. Anatomically guided Bayesian reconstruction (AMAP) and kernelized ordered subset expectation maximization (KEM) algorithms were implemented and compared against OSEM. Artificial lesions with a wide range of lesion-to-background contrasts were added to normal bone SPECT/CT studies. The quantitative accuracy was assessed by the error in lesion standardized uptake values and lesion detectability by the area under the receiver operating characteristic curve generated by a non-prewhitening matched filter. AMAP and KEM provided significantly better quantitative accuracy than OSEM at all contrast levels. Accuracy was the highest when SPECT lesions were matched to a lesion on CT. Correspondingly, AMAP and KEM also had significantly better lesion detectability than OSEM at all contrast levels and reconstructions with matching CT lesions performed the best. Quantitative differences between AMAP and KEM algorithms were minor. Visually AMAP and KEM images looked similar. Anatomically guided reconstruction improves lesion quantitation and detectability markedly compared to OSEM. Differences between AMAP and KEM algorithms were small and thus probably clinically insignificant.

Personalized synthetic MR imaging with deep learning enhancements

PURPOSE

Personalized synthetic MRI (syn-MRI) uses MR images of an individual subject acquired at a few design parameters (echo time, repetition time, flip angle) to obtain underlying parametric ( ρ , T 1 , T 2 ) $$ \left(\rho, {\mathrm{T}}_1,{\mathrm{T}}_2\right) $$ maps, from where MR images of that individual at other design parameter settings are synthesized. However, classical methods that use least-squares (LS) or maximum likelihood estimators (MLE) are unsatisfactory at higher noise levels because the underlying inverse problem is ill-posed. This article provides a pipeline to enhance the synthesis of such images in three-dimensional (3D) using a deep learning (DL) neural network architecture for spatial regularization in a personalized setting where having more than a few training images is impractical.

METHODS

Our DL enhancements employ a Deep Image Prior (DIP) with a U-net type denoising architecture that includes situations with minimal training data, such as personalized syn-MRI. We provide a general workflow for syn-MRI from three or more training images. Our workflow, called DIPsyn-MRI, uses DIP to enhance training images, then obtains parametric images using LS or MLE before synthesizing images at desired design parameter settings. DIPsyn-MRI is implemented in our publicly available Python package DeepSynMRI available at: https://github.com/StatPal/DeepSynMRI.

RESULTS

We demonstrate feasibility and improved performance of DIPsyn-MRI on 3D datasets acquired using the Brainweb interface for spin-echo and FLASH imaging sequences, at different noise levels. Our DL enhancements improve syn-MRI in the presence of different intensity nonuniformity levels of the magnetic field, for all but very low noise levels.

CONCLUSION

This article provides recipes and software to realistically facilitate DL-enhanced personalized syn-MRI.

Neural KEM: A Kernel Method With Deep Coefficient Prior for PET Image Reconstruction

Image reconstruction of low-count positron emission tomography (PET) data is challenging. Kernel methods address the challenge by incorporating image prior information in the forward model of iterative PET image reconstruction. The kernelized expectation-maximization (KEM) algorithm has been developed and demonstrated to be effective and easy to implement. A common approach for a further improvement of the kernel method would be adding an explicit regularization, which however leads to a complex optimization problem. In this paper, we propose an implicit regularization for the kernel method by using a deep coefficient prior, which represents the kernel coefficient image in the PET forward model using a convolutional neural-network. To solve the maximum-likelihood neural network-based reconstruction problem, we apply the principle of optimization transfer to derive a neural KEM algorithm. Each iteration of the algorithm consists of two separate steps: a KEM step for image update from the projection data and a deep-learning step in the image domain for updating the kernel coefficient image using the neural network. This optimization algorithm is guaranteed to monotonically increase the data likelihood. The results from computer simulations and real patient data have demonstrated that the neural KEM can outperform existing KEM and deep image prior methods.