Synergistic PET and SENSE MR Image Reconstruction Using Joint Sparsity Regularization

Bimodal PET/MRI generative reconstruction based on VAE architectures

Objective.In this study, we explore positron emission tomography (PET)/magnetic resonance imaging (MRI) joint reconstruction within a deep learning framework, introducing a novel synergistic method.Approach.We propose a new approach based on a variational autoencoder (VAE) constraint combined with the alternating direction method of multipliers (ADMM) optimization technique. We explore three VAE architectures, joint VAE, product of experts-VAE and multimodal JS divergence (MMJSD), to determine the optimal latent representation for the two modalities. We then trained and evaluated the architectures on a brain PET/MRI dataset.Main results.We showed that our approach takes advantage of each modality sharing information to each other, which results in improved peak signal-to-noise ratio and structural similarity as compared with traditional reconstruction, particularly for short acquisition times. We find that the one particular architecture, MMJSD, is the most effective for our methodology.Significance.The proposed method outperforms conventional approaches especially in noisy and undersampled conditions by making use of the two modalities together to compensate for the missing information.

Joint diffusion: mutual consistency-driven diffusion model for PET-MRI co-reconstruction

Objective. Positron Emission Tomography and Magnetic Resonance Imaging (PET-MRI) systems can obtain functional and anatomical scans. But PET suffers from a low signal-to-noise ratio, while MRI are time-consuming. To address time-consuming, an effective strategy involves reducing k-space data collection, albeit at the cost of lowering image quality. This study aims to leverage the inherent complementarity within PET-MRI data to enhance the image quality of PET-MRI.Approach. A novel PET-MRI joint reconstruction model, termed MC-Diffusion, is proposed in the Bayesian framework. The joint reconstruction problem is transformed into a joint regularization problem, where data fidelity terms of PET and MRI are expressed independently. The regular term, the derivative of the logarithm of the joint probability distribution of PET and MRI, employs a joint score-based diffusion model for learning. The diffusion model involves the forward diffusion process and the reverse diffusion process. The forward diffusion process adds noise to transform a complex joint data distribution into a known joint prior distribution for PET and MRI simultaneously, resembling a denoiser. The reverse diffusion process removes noise using a denoiser to revert the joint prior distribution to the original joint data distribution, effectively utilizing joint probability distribution to describe the correlations of PET and MRI for improved quality of joint reconstruction.Main results. Qualitative and quantitative improvements are observed with the MC-Diffusion model. Comparative analysis against LPLS and Joint ISAT-net on the ADNI dataset demonstrates superior performance by exploiting complementary information between PET and MRI. The MC-Diffusion model effectively enhances the quality of PET and MRI images.Significance. This study employs the MC-Diffusion model to enhance the quality of PET-MRI images by integrating the fundamental principles of PET and MRI modalities and leveraging their inherent complementarity. Furthermore, utilizing the diffusion model to learn the joint probability distribution of PET and MRI, thereby elucidating their latent correlation, facilitates a more profound comprehension of the priors obtained through deep learning, contrasting with black-box prior or artificially constructed structural similarities.

Clinical Value of Hybrid PET/MR Imaging: Brain Imaging Using PET/MR Imaging

Hybrid PET/MR imaging offers a unique opportunity to acquire MR imaging and PET information during a single imaging session. PET/MR imaging has numerous advantages, including enhanced diagnostic accuracy, improved disease characterization, and better treatment planning and monitoring. It enables the immediate integration of anatomic, functional, and metabolic imaging information, allowing for personalized characterization and monitoring of neurologic diseases. This review presents recent advances in PET/MR imaging and highlights advantages in clinical practice for neuro-oncology, epilepsy, and neurodegenerative disorders. PET/MR imaging provides valuable information about brain tumor metabolism, perfusion, and anatomic features, aiding in accurate delineation, treatment response assessment, and prognostication.

Semiology, EEG, and neuroimaging findings in temporal lobe epilepsies

Temporal lobe epilepsy (TLE) is the most common type of focal epilepsy. First descriptions of TLE date back in time and detailed portraits of epileptic seizures of temporal origin can be found in early medical reports as well as in the works of various artists and dramatists. Depending on the seizure onset zone, several subtypes of TLE have been identified, each one associated with peculiar ictal semiology. TLE can result from multiple etiological causes, ranging from genetic to lesional ones. While the diagnosis of TLE relies on detailed analysis of clinical as well as electroencephalographic (EEG) features, the lesions responsible for seizure generation can be highlighted by multiple brain imaging modalities or, in selected cases, by genetic investigations. TLE is the most common cause of refractory epilepsy and despite the great advances in diagnostic tools, no lesion is found in around one-third of patients. Surgical treatment is a safe and effective option, requiring presurgical investigations to accurately identify the seizure onset zone (SOZ). In selected cases, presurgical investigations need intracerebral investigations (such as stereoelectroencephalography) or dedicated metabolic imaging techniques (interictal PET and ictal SPECT) to correctly identify the brain structures to be removed.

Kernel graph filtering-A new method for dynamic sinogram denoising

Low count PET (positron emission tomography) imaging is often desirable in clinical diagnosis and biomedical research, but its images are generally very noisy, due to the very weak signals in the sinograms used in image reconstruction. To address this issue, this paper presents a novel kernel graph filtering method for dynamic PET sinogram denoising. This method is derived from treating the dynamic sinograms as the signals on a graph, and learning the graph adaptively from the kernel principal components of the sinograms to construct a lowpass kernel graph spectrum filter. The kernel graph filter thus obtained is then used to filter the original sinogram time frames to obtain the denoised sinograms for PET image reconstruction. Extensive tests and comparisons on the simulated and real life in-vivo dynamic PET datasets show that the proposed method outperforms the existing methods in sinogram denoising and image enhancement of dynamic PET at all count levels, especially at low count, with a great potential in real life applications of dynamic PET imaging.

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Learning non-local perfusion textures for high-quality computed tomography perfusion imaging

Background. Computed tomography perfusion (CTP) imaging plays a critical role in the acute stroke syndrome assessment due to its widespread availability, speed of image acquisition, and relatively low cost. However, due to its repeated scanning protocol, CTP imaging involves a substantial radiation dose, which might increase potential cancer risks.Methods. In this work, we present a novel deep learning model called non-local perfusion texture learning network (NPTN) for high-quality CTP imaging at low-dose cases. Specifically, considering abundant similarities in the CTP images, i.e. latent self-similarities within the non-local region in the CTP images, we firstly search the most similar pixels from the adjacent frames within a fixed search window to obtain the non-local similarities and to construct non-local textures vector. Then, both the low-dose frame and these non-local textures from adjacent frames are fed into a convolution neural network to predict high-quality CTP images, which can help better characterize the structure details and contrast variants in the targeted CTP image rather than simply utilizing the targeted frame itself. The residual learning strategy and batch normalization are utilized to boost the performance of the convolution neural network. In the experiment, the CTP images of 31 patients with suspected stroke disease are collected to demonstrate the performance of the presented NPTN method.Results. The results show the presented NPTN method obtains superior performance compared with the competing methods. From numerical value, at all dose levels, the presented NPTN method has achieved around 3.0 dB improvement of average PSNR, an increase of around 1.4% of average SSIM, and a decrease of around 4.8% of average RMSE in the low-dose CTP reconstruction task, and also has achieved an increase of around 3.4% of average SSIM and a decrease of around 61.1% of average RMSE in the cerebral blood flow (CBF) estimation task.Conclusions. The presented NPTN method can obtain high-quality CTP images and estimate high-accuracy CBF map by characterizing more structure details and contrast variants in the CTP image and outperform the competing methods at low-dose cases.

Quantitative PET in the 2020s: a roadmap

Positron emission tomography (PET) plays an increasingly important role in research and clinical applications, catalysed by remarkable technical advances and a growing appreciation of the need for reliable, sensitive biomarkers of human function in health and disease. Over the last 30 years, a large amount of the physics and engineering effort in PET has been motivated by the dominant clinical application during that period, oncology. This has led to important developments such as PET/CT, whole-body PET, 3D PET, accelerated statistical image reconstruction, and time-of-flight PET. Despite impressive improvements in image quality as a result of these advances, the emphasis on static, semi-quantitative 'hot spot' imaging for oncologic applications has meant that the capability of PET to quantify biologically relevant parameters based on tracer kinetics has not been fully exploited. More recent advances, such as PET/MR and total-body PET, have opened up the ability to address a vast range of new research questions, from which a future expansion of applications and radiotracers appears highly likely. Many of these new applications and tracers will, at least initially, require quantitative analyses that more fully exploit the exquisite sensitivity of PET and the tracer principle on which it is based. It is also expected that they will require more sophisticated quantitative analysis methods than those that are currently available. At the same time, artificial intelligence is revolutionizing data analysis and impacting the relationship between the statistical quality of the acquired data and the information we can extract from the data. In this roadmap, leaders of the key sub-disciplines of the field identify the challenges and opportunities to be addressed over the next ten years that will enable PET to realise its full quantitative potential, initially in research laboratories and, ultimately, in clinical practice.

Di-chromatic interpolation of magnetic resonance metabolic images

OBJECTIVE

Magnetic resonance imaging with hyperpolarized contrast agents can provide unprecedented in vivo measurements of metabolism, but yields images that are lower resolution than that achieved with proton anatomical imaging. In order to spatially localize the metabolic activity, the metabolic image must be interpolated to the size of the proton image. The most common methods for choosing the unknown values rely exclusively on values of the original uninterpolated image.

METHODS

In this work, we present an alternative method that uses the higher-resolution proton image to provide additional spatial structure. The interpolated image is the result of a convex optimization algorithm which is solved with the fast iterative shrinkage threshold algorithm (FISTA).

RESULTS

Results are shown with images of hyperpolarized pyruvate, lactate, and bicarbonate using data of the heart and brain from healthy human volunteers, a healthy porcine heart, and a human with prostate cancer.

Accelerated high-resolution free-breathing 3D whole-heart T2-prepared black-blood and bright-blood cardiovascular magnetic resonance

BACKGROUND

The free-breathing 3D whole-heart T2-prepared Bright-blood and black-blOOd phase SensiTive inversion recovery (BOOST) cardiovascular magnetic resonance (CMR) sequence was recently proposed for simultaneous bright-blood coronary CMR angiography and black-blood late gadolinium enhancement (LGE) imaging. This sequence enables simultaneous visualization of cardiac anatomy, coronary arteries and fibrosis. However, high-resolution (< 1.4 × 1.4 × 1.4 mm3) fully-sampled BOOST requires long acquisition times of ~ 20 min.

METHODS

In this work, we propose to extend a highly efficient respiratory-resolved motion-corrected reconstruction framework (XD-ORCCA) to T2-prepared BOOST to enable high-resolution 3D whole-heart coronary CMR angiography and black-blood LGE in a clinically feasible scan time. Twelve healthy subjects were imaged without contrast injection (pre-contrast BOOST) and 10 patients with suspected cardiovascular disease were imaged after contrast injection (post-contrast BOOST). A quantitative analysis software was used to compare accelerated pre-contrast BOOST against the fully-sampled counterpart (vessel sharpness and length of the left and right coronary arteries). Moreover, three cardiologists performed diagnostic image quality scoring for clinical 2D LGE and both bright- and black-blood 3D BOOST imaging using a 4-point scale (1-4, non-diagnostic-fully diagnostic). A two one-sided test of equivalence (TOST) was performed to compare the pre-contrast BOOST images. Nonparametric TOST was performed to compare post-contrast BOOST image quality scores.

RESULTS

The proposed method produces images from 3.8 × accelerated non-contrast-enhanced BOOST acquisitions with comparable vessel length and sharpness to those obtained from fully- sampled scans in healthy subjects. Moreover, in terms of visual grading, the 3D BOOST LGE datasets (median 4) and the clinical 2D counterpart (median 3.5) were found to be statistically equivalent (p < 0.05). In addition, bright-blood BOOST images allowed for visualization of the proximal and middle left anterior descending and right coronary sections with high diagnostic quality (mean score > 3.5).

CONCLUSIONS

The proposed framework provides high-resolution 3D whole-heart BOOST images from a single free-breathing acquisition in ~ 7 min.

Contribution of PET-MRI in brain diseases in clinical practice

PURPOSE OF REVIEW

Hybrid PET- MRI is a technique that has the ability to improve diagnostic accuracy in many applications, whereas PET and MRI performed separately often fail to provide accurate responses to clinical questions. Here, we review recent studies and current developments in PET-MRI, focusing on clinical applications.

RECENT FINDINGS

The combination of PET and MRI imaging methods aims at increasing the potential of each individual modality. Combined methods of image reconstruction and correction of PET-MRI attenuation are being developed, and a number of applications are being introduced into clinical practice. To date, the value of PET-MRI has been demonstrated for the evaluation of brain tumours in epilepsy and neurodegenerative diseases. Continued advances in data analysis regularly improve the efficiency and the potential application of multimodal biomarkers.

SUMMARY

PET-MRI provides simultaneous of anatomical, functional, biochemical and metabolic information for the personalized characterization and monitoring of neurological diseases. In this review, we show the advantage of the complementarity of different biomarkers obtained using PET-MRI data. We also present the recent advances made in this hybrid imaging modality and its advantages in clinical practice compared with MRI and PET separately.

A Robust Regularizer for Multiphase CT

Joint image reconstruction for multiphase CT can potentially improve image quality and reduce dose by leveraging the shared information among the phases. Multiphase CT scans are acquired sequentially. Inter-scan patient breathing causes small organ shifts and organ boundary misalignment among different phases. Existing multi-channel regularizers such as the joint total variation (TV) can introduce artifacts at misaligned organ boundaries. We propose a multi-channel regularizer using the infimal convolution (inf-conv) between a joint TV and a separable TV. It is robust against organ misalignment; it can work like a joint TV or a separable TV depending on a parameter setting. The effects of the parameter in the inf-conv regularizer are analyzed in detail. The properties of the inf-conv regularizer are then investigated numerically in a multi-channel image denoising setting. For algorithm implementation, the inf-conv regularizer is nonsmooth; inverse problems with the inf-conv regularizer can be solved using a number of primal-dual algorithms from nonsmooth convex minimization. Our numerical studies using synthesized 2-phase patient data and phantom data demonstrate that the inf-conv regularizer can largely maintain the advantages of the joint TV over the separable TV and reduce image artifacts of the joint TV due to organ misalignment.

Simultaneous use of individual and joint regularization terms in compressive sensing: Joint reconstruction of multi-channel multi-contrast MRI acquisitions

Multi-contrast images are commonly acquired together to maximize complementary diagnostic information, albeit at the expense of longer scan times. A time-efficient strategy to acquire high-quality multi-contrast images is to accelerate individual sequences and then reconstruct undersampled data with joint regularization terms that leverage common information across contrasts. However, these terms can cause features that are unique to a subset of contrasts to leak into the other contrasts. Such leakage-of-features may appear as artificial tissues, thereby misleading diagnosis. The goal of this study is to develop a compressive sensing method for multi-channel multi-contrast magnetic resonance imaging (MRI) that optimally utilizes shared information while preventing feature leakage. Joint regularization terms group sparsity and colour total variation are used to exploit common features across images while individual sparsity and total variation are also used to prevent leakage of distinct features across contrasts. The multi-channel multi-contrast reconstruction problem is solved via a fast algorithm based on Alternating Direction Method of Multipliers. The proposed method is compared against using only individual and only joint regularization terms in reconstruction. Comparisons were performed on single-channel simulated and multi-channel in-vivo datasets in terms of reconstruction quality and neuroradiologist reader scores. The proposed method demonstrates rapid convergence and improved image quality for both simulated and in-vivo datasets. Furthermore, while reconstructions that solely use joint regularization terms are prone to leakage-of-features, the proposed method reliably avoids leakage via simultaneous use of joint and individual terms, thereby holding great promise for clinical use.

Faster PET reconstruction with non-smooth priors by randomization and preconditioning

Uncompressed clinical data from modern positron emission tomography (PET) scanners are very large, exceeding 350 million data points (projection bins). The last decades have seen tremendous advancements in mathematical imaging tools many of which lead to non-smooth (i.e. non-differentiable) optimization problems which are much harder to solve than smooth optimization problems. Most of these tools have not been translated to clinical PET data, as the state-of-the-art algorithms for non-smooth problems do not scale well to large data. In this work, inspired by big data machine learning applications, we use advanced randomized optimization algorithms to solve the PET reconstruction problem for a very large class of non-smooth priors which includes for example total variation, total generalized variation, directional total variation and various different physical constraints. The proposed algorithm randomly uses subsets of the data and only updates the variables associated with these. While this idea often leads to divergent algorithms, we show that the proposed algorithm does indeed converge for any proper subset selection. Numerically, we show on real PET data (FDG and florbetapir) from a Siemens Biograph mMR that about ten projections and backprojections are sufficient to solve the MAP optimisation problem related to many popular non-smooth priors; thus showing that the proposed algorithm is fast enough to bring these models into routine clinical practice.

Multi-modal synergistic PET and MR reconstruction using mutually weighted quadratic priors

PURPOSE

To propose a framework for synergistic reconstruction of PET-MR and multi-contrast MR data to improve the image quality obtained from noisy PET data and from undersampled MR data.

THEORY AND METHODS

Weighted quadratic priors were devised to preserve common boundaries between PET-MR images while reducing noise, PET Gibbs ringing, and MR undersampling artifacts. These priors are iteratively reweighted using normalized multi-modal Gaussian similarity kernels. Synergistic PET-MR reconstructions were built on the PET maximum a posteriori expectation maximization algorithm and the MR regularized sensitivity encoding method. The proposed approach was compared to conventional methods, total variation, and prior-image weighted quadratic regularization methods. Comparisons were performed on a simulated [18 F]fluorodeoxyglucose-PET and T1 /T2 -weighted MR brain phantom, 2 in vivo T1 /T2 -weighted MR brain datasets, and an in vivo [18 F]fluorodeoxyglucose-PET and fluid-attenuated inversion recovery/T1 -weighted MR brain dataset.

RESULTS

Simulations showed that synergistic reconstructions achieve the lowest quantification errors for all image modalities compared to conventional, total variation, and weighted quadratic methods. Whereas total variation regularization preserved modality-unique features, this method failed to recover PET details and was not able to reduce MR artifacts compared to our proposed method. For in vivo MR data, our method maintained similar image quality for 3× and 14× accelerated data. Reconstruction of the PET-MR dataset also demonstrated improved performance of our method compared to the conventional independent methods in terms of reduced Gibbs and undersampling artifacts.

CONCLUSION

The proposed methodology offers a robust multi-modal synergistic image reconstruction framework that can be readily built on existing established algorithms.

Effect of PET-MR Inconsistency in the Kernel Image Reconstruction Method

Anatomically-driven image reconstruction algorithms have become very popular in positron emission tomography (PET) where they have demonstrated improved image resolution and quantification. This work, consider the effect of spatial inconsistency between MR and PET images in hot and cold regions of the PET image. We investigate these effects on the kernel method from machine learning, in particular, the hybrid kernelized expectation maximization (HKEM). These were applied to Jaszczak phantom and patient data acquired with the Biograph Siemens mMR. The results show that even a small shift can cause a significant change in activity concentration. In general, the PET-MR inconsistencies can induce the partial volume effect, more specifically the 'spill-in' of the affected cold regions and the 'spill-out' from the hot regions. The maximum change was about 100% for the cold region and 10% for the hot lesion using KEM, against the 37% and 8% obtained with HKEM. The findings of this work suggest that including PET information in the kernel enhances the flexibility of the reconstruction in case of spatial inconsistency. Nevertheless, accurate registration and choice of the appropriate MR image for the creation of the kernel is essential to avoid artifacts, blurring, and bias.

From simultaneous to synergistic MR-PET brain imaging: A review of hybrid MR-PET imaging methodologies

Simultaneous Magnetic Resonance Imaging (MRI) and Positron Emission Tomography (PET) scanning is a recent major development in biomedical imaging. The full integration of the PET detector ring and electronics within the MR system has been a technologically challenging design to develop but provides capacity for simultaneous imaging and the potential for new diagnostic and research capability. This article reviews state-of-the-art MR-PET hardware and software, and discusses future developments focusing on neuroimaging methodologies for MR-PET scanning. We particularly focus on the methodologies that lead to an improved synergy between MRI and PET, including optimal data acquisition, PET attenuation and motion correction, and joint image reconstruction and processing methods based on the underlying complementary and mutual information. We further review the current and potential future applications of simultaneous MR-PET in both systems neuroscience and clinical neuroimaging research. We demonstrate a simultaneous data acquisition protocol to highlight new applications of MR-PET neuroimaging research studies.

Penalized maximum likelihood simultaneous longitudinal PET image reconstruction with difference-image priors

PURPOSE

Many clinical contexts require the acquisition of multiple positron emission tomography (PET) scans of a single subject, for example, to observe and quantitate changes in functional behaviour in tumors after treatment in oncology. Typically, the datasets from each of these scans are reconstructed individually, without exploiting the similarities between them. We have recently shown that sharing information between longitudinal PET datasets by penalizing voxel-wise differences during image reconstruction can improve reconstructed images by reducing background noise and increasing the contrast-to-noise ratio of high-activity lesions. Here, we present two additional novel longitudinal difference-image priors and evaluate their performance using two-dimesional (2D) simulation studies and a three-dimensional (3D) real dataset case study.

METHODS

We have previously proposed a simultaneous difference-image-based penalized maximum likelihood (PML) longitudinal image reconstruction method that encourages sparse difference images (DS-PML), and in this work we propose two further novel prior terms. The priors are designed to encourage longitudinal images with corresponding differences which have (a) low entropy (DE-PML), and (b) high sparsity in their spatial gradients (DTV-PML). These two new priors and the originally proposed longitudinal prior were applied to 2D-simulated treatment response [¹⁸ F]fluorodeoxyglucose (FDG) brain tumor datasets and compared to standard maximum likelihood expectation-maximization (MLEM) reconstructions. These 2D simulation studies explored the effects of penalty strengths, tumor behaviour, and interscan coupling on reconstructed images. Finally, a real two-scan longitudinal data series acquired from a head and neck cancer patient was reconstructed with the proposed methods and the results compared to standard reconstruction methods.

RESULTS

Using any of the three priors with an appropriate penalty strength produced images with noise levels equivalent to those seen when using standard reconstructions with increased counts levels. In tumor regions, each method produces subtly different results in terms of preservation of tumor quantitation and reconstruction root mean-squared error (RMSE). In particular, in the two-scan simulations, the DE-PML method produced tumor means in close agreement with MLEM reconstructions, while the DTV-PML method produced the lowest errors due to noise reduction within the tumor. Across a range of tumor responses and different numbers of scans, similar results were observed, with DTV-PML producing the lowest errors of the three priors and DE-PML producing the lowest bias. Similar improvements were observed in the reconstructions of the real longitudinal datasets, although imperfect alignment of the two PET images resulted in additional changes in the difference image that affected the performance of the proposed methods.

CONCLUSION

Reconstruction of longitudinal datasets by penalizing difference images between pairs of scans from a data series allows for noise reduction in all reconstructed images. An appropriate choice of penalty term and penalty strength allows for this noise reduction to be achieved while maintaining reconstruction performance in regions of change, either in terms of quantitation of mean intensity via DE-PML, or in terms of tumor RMSE via DTV-PML. Overall, improving the image quality of longitudinal datasets via simultaneous reconstruction has the potential to improve upon currently used methods, allow dose reduction, or reduce scan time while maintaining image quality at current levels.