PET Image Deblurring and Super-Resolution with an MR-Based Joint Entropy Prior

Super-resolution techniques for biomedical applications and challenges

Super-resolution (SR) techniques have revolutionized the field of biomedical applications by detailing the structures at resolutions beyond the limits of imaging or measuring tools. These techniques have been applied in various biomedical applications, including microscopy, magnetic resonance imaging (MRI), computed tomography (CT), X-ray, electroencephalogram (EEG), ultrasound, etc. SR methods are categorized into two main types: traditional non-learning-based methods and modern learning-based approaches. In both applications, SR methodologies have been effectively utilized on biomedical images, enhancing the visualization of complex biological structures. Additionally, these methods have been employed on biomedical data, leading to improvements in computational precision and efficiency for biomedical simulations. The use of SR techniques has resulted in more detailed and accurate analyses in diagnostics and research, essential for early disease detection and treatment planning. However, challenges such as computational demands, data interpretation complexities, and the lack of unified high-quality data persist. The article emphasizes these issues, underscoring the need for ongoing development in SR technologies to further improve biomedical research and patient care outcomes.

Neural blind deconvolution for deblurring and supersampling PSMA PET

Objective. To simultaneously deblur and supersample prostate specific membrane antigen (PSMA) positron emission tomography (PET) images using neural blind deconvolution.Approach. Blind deconvolution is a method of estimating the hypothetical 'deblurred' image along with the blur kernel (related to the point spread function) simultaneously. Traditionalmaximum a posterioriblind deconvolution methods require stringent assumptions and suffer from convergence to a trivial solution. A method of modelling the deblurred image and kernel with independent neural networks, called 'neural blind deconvolution' had demonstrated success for deblurring 2D natural images in 2020. In this work, we adapt neural blind deconvolution to deblur PSMA PET images while simultaneous supersampling to double the original resolution. We compare this methodology with several interpolation methods in terms of resultant blind image quality metrics and test the model's ability to predict accurate kernels by re-running the model after applying artificial 'pseudokernels' to deblurred images. The methodology was tested on a retrospective set of 30 prostate patients as well as phantom images containing spherical lesions of various volumes.Main results. Neural blind deconvolution led to improvements in image quality over other interpolation methods in terms of blind image quality metrics, recovery coefficients, and visual assessment. Predicted kernels were similar between patients, and the model accurately predicted several artificially-applied pseudokernels. Localization of activity in phantom spheres was improved after deblurring, allowing small lesions to be more accurately defined.Significance. The intrinsically low spatial resolution of PSMA PET leads to partial volume effects (PVEs) which negatively impact uptake quantification in small regions. The proposed method can be used to mitigate this issue, and can be straightforwardly adapted for other imaging modalities.

Quasi-supervised learning for super-resolution PET

Low resolution of positron emission tomography (PET) limits its diagnostic performance. Deep learning has been successfully applied to achieve super-resolution PET. However, commonly used supervised learning methods in this context require many pairs of low- and high-resolution (LR and HR) PET images. Although unsupervised learning utilizes unpaired images, the results are not as good as that obtained with supervised deep learning. In this paper, we propose a quasi-supervised learning method, which is a new type of weakly-supervised learning methods, to recover HR PET images from LR counterparts by leveraging similarity between unpaired LR and HR image patches. Specifically, LR image patches are taken from a patient as inputs, while the most similar HR patches from other patients are found as labels. The similarity between the matched HR and LR patches serves as a prior for network construction. Our proposed method can be implemented by designing a new network or modifying an existing network. As an example in this study, we have modified the cycle-consistent generative adversarial network (CycleGAN) for super-resolution PET. Our numerical and experimental results qualitatively and quantitatively show the merits of our method relative to the state-of-the-art methods. The code is publicly available at https://github.com/PigYang-ops/CycleGAN-QSDL.

State-of-the-art imaging of neuromodulatory subcortical systems in aging and Alzheimer's disease: Challenges and opportunities

Primary prevention trials have shifted their focus to the earliest stages of Alzheimer's disease (AD). Autopsy data indicates that the neuromodulatory subcortical systems' (NSS) nuclei are specifically vulnerable to initial tau pathology, indicating that these nuclei hold great promise for early detection of AD in the context of the aging brain. The increasing availability of new imaging methods, ultra-high field scanners, new radioligands, and routine deep brain stimulation implants has led to a growing number of NSS neuroimaging studies on aging and neurodegeneration. Here, we review findings of current state-of-the-art imaging studies assessing the structure, function, and molecular changes of these nuclei during aging and AD. Furthermore, we identify the challenges associated with these imaging methods, important pathophysiologic gaps to fill for the AD NSS neuroimaging field, and provide future directions to improve our assessment, understanding, and clinical use of in vivo imaging of the NSS.

Dynamic PET images denoising using spectral graph wavelet transform

Positron emission tomography (PET) is a non-invasive molecular imaging method for quantitative observation of physiological and biochemical changes in living organisms. The quality of the reconstructed PET image is limited by many different physical degradation factors. Various denoising methods including Gaussian filtering (GF) and non-local mean (NLM) filtering have been proposed to improve the image quality. However, image denoising usually blurs edges, of which high frequency components are filtered as noises. On the other hand, it is well-known that edges in a PET image are important to detection and recognition of a lesion. Denoising while preserving the edges of PET images remains an important yet challenging problem in PET image processing. In this paper, we propose a novel denoising method with good edge-preserving performance based on spectral graph wavelet transform (SGWT) for dynamic PET images denoising. We firstly generate a composite image from the entire time series, then perform SGWT on the PET images, and finally reconstruct the low graph frequency content to get the denoised dynamic PET images. Experimental results on simulation and in vivo data show that the proposed approach significantly outperforms the GF, NLM and graph filtering methods. Compared with deep learning-based method, the proposed method has the similar denoising performance, but it does not need lots of training data and has low computational complexity.

A new approach to overcome the inconsistency between SPECT and the anatomical map in maximum A-posterior expectation-maximization reconstruction algorithm

Noise reduction while preserving spatial resolution is one of the most important challenges in the reconstructing of emission tomography images. One of the resolving methods is the Bowsher maximum a-posteriori expectation-maximization reconstruction (MAPEM) algorithm. This method considers a binary selection of the neighbors of each voxel based on the prior anatomical values to use in the regularization function. This method is particularly susceptible to imposing the wrong data into the reconstructed image due to the spatial or functional inconsistencies between the anatomical image and the actual activity distribution. Because of the poor spatial resolution of single-photon emission tomography (SPECT) images and the different nature of emission and anatomical imaging, there is not enough certainty of inconsistency with anatomical images. Therefore, we proposed a new weighted Bowsher method that can overcome this weakness while the image quality indexes, especially the spatial resolution, are almost preserved. In the proposed method, each of the neighbors of a specific voxel takes a constant weight depending on the order of its value and independent of its intensity quantity. The proposed method was evaluated using some different physical phantoms and a patient scan. The results show that the proposed method has superiority in the presence of inconsistency; moreover, the proposed method gives nearly similar results to the regular Bowsher MAPEM in case of consistency. In conclusion, we show that using a suitable constant weighting factor in Bowsher MAPEM, one can operatively reduce the image noise while preserving the image quality parameters where the emission tomography images are either consistent or inconsistent with the prior anatomical map.

Effect of Denoising and Deblurring 18F-Fluorodeoxyglucose Positron Emission Tomography Images on a Deep Learning Model’s Classification Performance for Alzheimer’s Disease

Alzheimer’s disease (AD) is the most common progressive neurodegenerative disease. 18F-fluorodeoxyglucose positron emission tomography (18F-FDG PET) is widely used to predict AD using a deep learning model. However, the effects of noise and blurring on 18F-FDG PET images were not considered. The performance of a classification model trained using raw, deblurred (by the fast total variation deblurring method), or denoised (by the median modified Wiener filter) 18F-FDG PET images without or with cropping around the limbic system area using a 3D deep convolutional neural network was investigated. The classification model trained using denoised whole-brain 18F-FDG PET images achieved classification performance (0.75/0.65/0.79/0.39 for sensitivity/specificity/F1-score/Matthews correlation coefficient (MCC), respectively) higher than that with raw and deblurred 18F-FDG PET images. The classification model trained using cropped raw 18F-FDG PET images achieved higher performance (0.78/0.63/0.81/0.40 for sensitivity/specificity/F1-score/MCC) than the whole-brain 18F-FDG PET images (0.72/0.32/0.71/0.10 for sensitivity/specificity/F1-score/MCC, respectively). The 18F-FDG PET image deblurring and cropping (0.89/0.67/0.88/0.57 for sensitivity/specificity/F1-score/MCC) procedures were the most helpful for improving performance. For this model, the right middle frontal, middle temporal, insula, and hippocampus areas were the most predictive of AD using the class activation map. Our findings demonstrate that 18F-FDG PET image preprocessing and cropping improves the explainability and potential clinical applicability of deep learning models.

Artificial Intelligence-Based Image Enhancement in PET Imaging: Noise Reduction and Resolution Enhancement

High noise and low spatial resolution are two key confounding factors that limit the qualitative and quantitative accuracy of PET images. Artificial intelligence models for image denoising and deblurring are becoming increasingly popular for the postreconstruction enhancement of PET images. We present a detailed review of recent efforts for artificial intelligence-based PET image enhancement with a focus on network architectures, data types, loss functions, and evaluation metrics. We also highlight emerging areas in this field that are quickly gaining popularity, identify barriers to large-scale adoption of artificial intelligence models for PET image enhancement, and discuss future directions.

Towards lower-dose PET using physics-based uncertainty-aware multimodal learning with robustness to out-of-distribution data

Radiation exposure in positron emission tomography (PET) imaging limits its usage in the studies of radiation-sensitive populations, e.g., pregnant women, children, and adults that require longitudinal imaging. Reducing the PET radiotracer dose or acquisition time reduces photon counts, which can deteriorate image quality. Recent deep-neural-network (DNN) based methods for image-to-image translation enable the mapping of low-quality PET images (acquired using substantially reduced dose), coupled with the associated magnetic resonance imaging (MRI) images, to high-quality PET images. However, such DNN methods focus on applications involving test data that match the statistical characteristics of the training data very closely and give little attention to evaluating the performance of these DNNs on new out-of-distribution (OOD) acquisitions. We propose a novel DNN formulation that models the (i) underlying sinogram-based physics of the PET imaging system and (ii) the uncertainty in the DNN output through the per-voxel heteroscedasticity of the residuals between the predicted and the high-quality reference images. Our sinogram-based uncertainty-aware DNN framework, namely, suDNN, estimates a standard-dose PET image using multimodal input in the form of (i) a low-dose/low-count PET image and (ii) the corresponding multi-contrast MRI images, leading to improved robustness of suDNN to OOD acquisitions. Results on in vivo simultaneous PET-MRI, and various forms of OOD data in PET-MRI, show the benefits of suDNN over the current state of the art, quantitatively and qualitatively.

Populational and individual information based PET image denoising using conditional unsupervised learning

Our study aims to improve the signal-to-noise ratio of positron emission tomography (PET) imaging using conditional unsupervised learning. The proposed method does not require low- and high-quality pairs for network training which can be easily applied to existing PET/computed tomography (CT) and PET/magnetic resonance (MR) datasets. This method consists of two steps: populational training and individual fine-tuning. As for populational training, a network was first pre-trained by a group of patients' noisy PET images and the corresponding anatomical prior images from CT or MR. As for individual fine-tuning, a new network with initial parameters inherited from the pre-trained network was fine-tuned by the test patient's noisy PET image and the corresponding anatomical prior image. Only the last few layers were fine-tuned to take advantage of the populational information and the pre-training efforts. Both networks shared the same structure and took the CT or MR images as the network input so that the network output was conditioned on the patient's anatomic prior information. The noisy PET images were used as the training and fine-tuning labels. The proposed method was evaluated on a⁶⁸Ga-PPRGD2 PET/CT dataset and a¹⁸F-FDG PET/MR dataset. For the PET/CT dataset, with the original noisy PET image as the baseline, the proposed method has a significantly higher contrast-to noise ratio (CNR) improvement (71.85% ± 27.05%) than Gaussian (12.66% ± 6.19%,P= 0.002), nonlocal mean method (22.60% ± 13.11%,P= 0.002) and conditional deep image prior method (52.94% ± 21.79%,P= 0.0039). For the PET/MR dataset, compared to Gaussian (18.73% ± 9.98%,P< 0.0001), NLM (26.01% ± 19.40%,P< 0.0001) and CDIP (47.48% ± 25.36%,P< 0.0001), the CNR improvement ratio of the proposed method (58.07% ± 28.45%) is the highest. In addition, the denoised images using both datasets also showed that the proposed method can accurately restore tumor structures while also smoothing out the noise.

Longitudinal predictive modeling of tau progression along the structural connectome

Tau neurofibrillary tangles, a pathophysiological hallmark of Alzheimer's disease (AD), exhibit a stereotypical spatiotemporal trajectory that is strongly correlated with disease progression and cognitive decline. Personalized prediction of tau progression is, therefore, vital for the early diagnosis and prognosis of AD. Evidence from both animal and human studies is suggestive of tau transmission along the brains preexisting neural connectivity conduits. We present here an analytic graph diffusion framework for individualized predictive modeling of tau progression along the structural connectome. To account for physiological processes that lead to active generation and clearance of tau alongside passive diffusion, our model uses an inhomogenous graph diffusion equation with a source term and provides closed-form solutions to this equation for linear and exponential source functionals. Longitudinal imaging data from two cohorts, the Harvard Aging Brain Study (HABS) and the Alzheimer's Disease Neuroimaging Initiative (ADNI), were used to validate the model. The clinical data used for developing and validating the model include regional tau measures extracted from longitudinal positron emission tomography (PET) scans based on the ¹⁸F-Flortaucipir radiotracer and individual structural connectivity maps computed from diffusion tensor imaging (DTI) by means of tractography and streamline counting. Two-timepoint tau PET scans were used to assess the goodness of model fit. Three-timepoint tau PET scans were used to assess predictive accuracy via comparison of predicted and observed tau measures at the third timepoint. Our results show high consistency between predicted and observed tau and differential tau from region-based analysis. While the prognostic value of this approach needs to be validated in a larger cohort, our preliminary results suggest that our longitudinal predictive model, which offers an in vivo macroscopic perspective on tau progression in the brain, is potentially promising as a personalizable predictive framework for AD.

Incorporation of anatomical MRI knowledge for enhanced mapping of brain metabolism using functional PET

Functional positron emission tomography (fPET) imaging using continuous infusion of [18F]-fluorodeoxyglucose (FDG) is a novel neuroimaging technique to track dynamic glucose utilization in the brain. In comparison to conventional static or dynamic bolus PET, fPET maintains a sustained supply of glucose in the blood plasma which improves sensitivity to measure dynamic glucose changes in the brain, and enables mapping of dynamic brain activity in task-based and resting-state fPET studies. However, there is a trade-off between temporal resolution and spatial noise due to the low concentration of FDG and the limited sensitivity of multi-ring PET scanners. Images from fPET studies suffer from partial volume errors and residual scatter noise that may cause the cerebral metabolic functional maps to be biased. Gaussian smoothing filters used to denoise the fPET images are suboptimal, as they introduce additional partial volume errors. In this work, a post-processing framework based on a magnetic resonance (MR) Bowsher-like prior was used to improve the spatial and temporal signal to noise characteristics of the fPET images. The performance of the MR guided method was compared with conventional denosing methods using both simulated and in vivo task fPET datasets. The results demonstrate that the MR-guided fPET framework denoises the fPET images and improves the partial volume correction, consequently enhancing the sensitivity to identify brain activation, and improving the anatomical accuracy for mapping changes of brain metabolism in response to a visual stimulation task. The framework extends the use of functional PET to investigate the dynamics of brain metabolic responses for faster presentation of brain activation tasks, and for applications in low dose PET imaging.

CT-less Direct Correction of Attenuation and Scatter in the Image Space Using Deep Learning for Whole-Body FDG PET: Potential Benefits and Pitfalls

PURPOSE

To demonstrate the feasibility of CT-less attenuation and scatter correction (ASC) in the image space using deep learning for whole-body PET, with a focus on the potential benefits and pitfalls.

MATERIALS AND METHODS

In this retrospective study, 110 whole-body fluorodeoxyglucose (FDG) PET/CT studies acquired in 107 patients (mean age ± standard deviation, 58 years ± 18; age range, 11-92 years; 72 females) from February 2016 through January 2018 were randomly collected. A total of 37.3% (41 of 110) of the studies showed metastases, with diverse FDG PET findings throughout the whole body. A U-Net-based network was developed for directly transforming noncorrected PET (PET_NC) into attenuation- and scatter-corrected PET (PET_ASC). Deep learning-corrected PET (PET_DL) images were quantitatively evaluated by using the standardized uptake value (SUV) of the normalized root mean square error, the peak signal-to-noise ratio, and the structural similarity index, in addition to a joint histogram for statistical analysis. Qualitative reviews by radiologists revealed the potential benefits and pitfalls of this correction method.

RESULTS

The normalized root mean square error (0.21 ± 0.05 [mean SUV ± standard deviation]), mean peak signal-to-noise ratio (36.3 ± 3.0), mean structural similarity index (0.98 ± 0.01), and voxelwise correlation (97.62%) of PET_DL demonstrated quantitatively high similarity with PET_ASC. Radiologist reviews revealed the overall quality of PET_DL. The potential benefits of PET_DL include a radiation dose reduction on follow-up scans and artifact removal in the regions with attenuation correction- and scatter correction-based artifacts. The pitfalls involve potential false-negative results due to blurring or missing lesions or false-positive results due to pseudo-low-uptake patterns.

CONCLUSION

Deep learning-based direct ASC at whole-body PET is feasible and potentially can be used to overcome the current limitations of CT-based approaches, benefiting patients who are sensitive to radiation from CT.Supplemental material is available for this article.© RSNA, 2020.

Parameter-Transferred Wasserstein Generative Adversarial Network (PT-WGAN) for Low-Dose PET Image Denoising

Due to the widespread use of positron emission tomography (PET) in clinical practice, the potential risk of PET-associated radiation dose to patients needs to be minimized. However, with the reduction in the radiation dose, the resultant images may suffer from noise and artifacts that compromise diagnostic performance. In this paper, we propose a parameter-transferred Wasserstein generative adversarial network (PT-WGAN) for low-dose PET image denoising. The contributions of this paper are twofold: i) a PT-WGAN framework is designed to denoise low-dose PET images without compromising structural details, and ii) a task-specific initialization based on transfer learning is developed to train PT-WGAN using trainable parameters transferred from a pretrained model, which significantly improves the training efficiency of PT-WGAN. The experimental results on clinical data show that the proposed network can suppress image noise more effectively while preserving better image fidelity than recently published state-of-the-art methods. We make our code available at https://github.com/90n9-yu/PT-WGAN.

PET image super-resolution using generative adversarial networks

The intrinsically low spatial resolution of positron emission tomography (PET) leads to image quality degradation and inaccurate image-based quantitation. Recently developed supervised super-resolution (SR) approaches are of great relevance to PET but require paired low- and high-resolution images for training, which are usually unavailable for clinical datasets. In this paper, we present a self-supervised SR (SSSR) technique for PET based on dual generative adversarial networks (GANs), which precludes the need for paired training data, ensuring wider applicability and adoptability. The SSSR network receives as inputs a low-resolution PET image, a high-resolution anatomical magnetic resonance (MR) image, spatial information (axial and radial coordinates), and a high-dimensional feature set extracted from an auxiliary CNN which is separately-trained in a supervised manner using paired simulation datasets. The network is trained using a loss function which includes two adversarial loss terms, a cycle consistency term, and a total variation penalty on the SR image. We validate the SSSR technique using a clinical neuroimaging dataset. We demonstrate that SSSR is promising in terms of image quality, peak signal-to-noise ratio, structural similarity index, contrast-to-noise ratio, and an additional no-reference metric developed specifically for SR image quality assessment. Comparisons with other SSSR variants suggest that its high performance is largely attributable to simulation guidance.

Super-Resolution PET Imaging Using Convolutional Neural Networks

Positron emission tomography (PET) suffers from severe resolution limitations which reduce its quantitative accuracy. In this paper, we present a super-resolution (SR) imaging technique for PET based on convolutional neural networks (CNNs). To facilitate the resolution recovery process, we incorporate high-resolution (HR) anatomical information based on magnetic resonance (MR) imaging. We introduce the spatial location information of the input image patches as additional CNN inputs to accommodate the spatially-variant nature of the blur kernels in PET. We compared the performance of shallow (3-layer) and very deep (20-layer) CNNs with various combinations of the following inputs: low-resolution (LR) PET, radial locations, axial locations, and HR MR. To validate the CNN architectures, we performed both realistic simulation studies using the BrainWeb digital phantom and clinical studies using neuroimaging datasets. For both simulation and clinical studies, the LR PET images were based on the Siemens HR+ scanner. Two different scenarios were examined in simulation: one where the target HR image is the ground-truth phantom image and another where the target HR image is based on the Siemens HRRT scanner - a high-resolution dedicated brain PET scanner. The latter scenario was also examined using clinical neuroimaging datasets. A number of factors affected relative performance of the different CNN designs examined, including network depth, target image quality, and the resemblance between the target and anatomical images. In general, however, all deep CNNs outperformed classical penalized deconvolution and partial volume correction techniques by large margins both qualitatively (e.g., edge and contrast recovery) and quantitatively (as indicated by three metrics: peak signal-to-noise-ratio, structural similarity index, and contrast-to-noise ratio).

PET image denoising using unsupervised deep learning

PURPOSE

Image quality of positron emission tomography (PET) is limited by various physical degradation factors. Our study aims to perform PET image denoising by utilizing prior information from the same patient. The proposed method is based on unsupervised deep learning, where no training pairs are needed.

METHODS

In this method, the prior high-quality image from the patient was employed as the network input and the noisy PET image itself was treated as the training label. Constrained by the network structure and the prior image input, the network was trained to learn the intrinsic structure information from the noisy image and output a restored PET image. To validate the performance of the proposed method, a computer simulation study based on the BrainWeb phantom was first performed. A ⁶⁸Ga-PRGD2 PET/CT dataset containing 10 patients and a ¹⁸F-FDG PET/MR dataset containing 30 patients were later on used for clinical data evaluation. The Gaussian, non-local mean (NLM) using CT/MR image as priors, BM4D, and Deep Decoder methods were included as reference methods. The contrast-to-noise ratio (CNR) improvements were used to rank different methods based on Wilcoxon signed-rank test.

RESULTS

For the simulation study, contrast recovery coefficient (CRC) vs. standard deviation (STD) curves showed that the proposed method achieved the best performance regarding the bias-variance tradeoff. For the clinical PET/CT dataset, the proposed method achieved the highest CNR improvement ratio (53.35% ± 21.78%), compared with the Gaussian (12.64% ± 6.15%, P = 0.002), NLM guided by CT (24.35% ± 16.30%, P = 0.002), BM4D (38.31% ± 20.26%, P = 0.002), and Deep Decoder (41.67% ± 22.28%, P = 0.002) methods. For the clinical PET/MR dataset, the CNR improvement ratio of the proposed method achieved 46.80% ± 25.23%, higher than the Gaussian (18.16% ± 10.02%, P < 0.0001), NLM guided by MR (25.36% ± 19.48%, P < 0.0001), BM4D (37.02% ± 21.38%, P < 0.0001), and Deep Decoder (30.03% ± 20.64%, P < 0.0001) methods. Restored images for all the datasets demonstrate that the proposed method can effectively smooth out the noise while recovering image details.

CONCLUSION

The proposed unsupervised deep learning framework provides excellent image restoration effects, outperforming the Gaussian, NLM methods, BM4D, and Deep Decoder methods.