Vector-valued image processing by parallel level sets

Enhancing ultrasonic attenuation images through multi-frequency coupling with total nuclear variation

Quantitative ultrasound is a non-invasive image modality that numerically characterizes tissues for medical diagnosis using acoustical parameters, such as the attenuation coefficient slope. A previous study introduced the total variation spectral log difference (TVSLD) method, which denoises spectral log ratios on a single-channel basis without inter-channel coupling. Therefore, this work proposes a multi-frequency joint framework by coupling information across frequency channels exploiting structural similarities among the spectral ratios to increase the quality of the attenuation images. A modification based on the total nuclear variation (TNV) was considered. Metrics were compared to the TVSLD method with simulated and experimental phantoms and two samples of fibroadenoma in vivo breast tissue. The TNV demonstrated superior performance, yielding enhanced attenuation coefficient slope maps with fewer artifacts at boundaries and a stable error. In terms of the contrast-to-noise ratio enhancement, the TNV approach obtained an average percentage improvement of 34% in simulation, 38% in the experimental phantom, and 89% in two in vivo breast tissue samples compared to TVSLD, showing potential to enhance visual clarity and depiction of attenuation images.

Joint motion estimation and penalized image reconstruction algorithm with anatomical priors for gated TOF-PET/CT

The presence of respiratory motion not only degrades the reconstructed image but also limits the utilization of anatomical priors in emission tomography. In this study, we explore the potential application of a joint motion estimation and penalized image reconstruction algorithm using anatomical priors in gated time-of-flight positron emission tomography/computed tomography (PET/CT). The algorithm is able to warp both the activity image and the attenuation map to align them with the measured data with the facilitation of anatomical information contained in the attenuation map. Five patient datasets, three acquired in single-bed position and two acquired in whole-body continuous-bed-motion mode, are included. For each patient, the attenuation map is derived from a breath-hold CT. The Parallel Levels Sets (PLS) is chosen as a representative anatomical prior. In addition to demonstrating the reliability of the estimated motion and the benefits of incorporating anatomical prior, preliminary results also indicate that the algorithm shows the potential to reconstruct an activity image in the space corresponding to that of the attenuation map, which could be applied to address the potential misalignment issue in applications involving multiple PET acquisitions but a single CT.

Core Imaging Library - Part II: multichannel reconstruction for dynamic and spectral tomography

The newly developed core imaging library (CIL) is a flexible plug and play library for tomographic imaging with a specific focus on iterative reconstruction. CIL provides building blocks for tailored regularized reconstruction algorithms and explicitly supports multichannel tomographic data. In the first part of this two-part publication, we introduced the fundamentals of CIL. This paper focuses on applications of CIL for multichannel data, e.g. dynamic and spectral. We formalize different optimization problems for colour processing, dynamic and hyperspectral tomography and demonstrate CIL's capabilities for designing state-of-the-art reconstruction methods through case studies and code snapshots. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'.

Synergistic multi-spectral CT reconstruction with directional total variation

This work considers synergistic multi-spectral CT reconstruction where information from all available energy channels is combined to improve the reconstruction of each individual channel. We propose to fuse these available data (represented by a single sinogram) to obtain a polyenergetic image which keeps structural information shared by the energy channels with increased signal-to-noise ratio. This new image is used as prior information during a channel-by-channel minimization process through the directional total variation. We analyse the use of directional total variation within variational regularization and iterative regularization. Our numerical results on simulated and experimental data show improvements in terms of image quality and in computational speed. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 2'.

(An overview of) Synergistic reconstruction for multimodality/multichannel imaging methods

Imaging is omnipresent in modern society with imaging devices based on a zoo of physical principles, probing a specimen across different wavelengths, energies and time. Recent years have seen a change in the imaging landscape with more and more imaging devices combining that which previously was used separately. Motivated by these hardware developments, an ever increasing set of mathematical ideas is appearing regarding how data from different imaging modalities or channels can be synergistically combined in the image reconstruction process, exploiting structural and/or functional correlations between the multiple images. Here we review these developments, give pointers to important challenges and provide an outlook as to how the field may develop in the forthcoming years. This article is part of the theme issue 'Synergistic tomographic image reconstruction: part 1'.

Gradient in spectral and color images: from the Di Zenzo initial construction to a generic proposition

In this article, we define a generic gradient for color and spectral images, considering a proposed taxonomy of the state of the art. A full-vector gradient, taking into account the sensor's characteristics, is in compliance with the metrological properties of genericity, robustness, and reproducibility. Here, we construct a protocol to compare gradients from different sensors. The comparison is developed by simulating sensors using their spectral characteristics. We develop three experiments using this protocol. The first experiment shows the consistency of results for similar sensors; the second demonstrates the genericity of the approach, adapted to any kind of imaging sensors; and the third focuses on the channel inter-correlation considering sensors such as in the color vision deficiency case.

Intercomparison of MR-informed PET image reconstruction methods

PURPOSE

Numerous image reconstruction methodologies for positron emission tomography (PET) have been developed that incorporate magnetic resonance (MR) imaging structural information, producing reconstructed images with improved suppression of noise and reduced partial volume effects. However, the influence of MR structural information also increases the possibility of suppression or bias of structures present only in the PET data (PET-unique regions). To address this, further developments for MR-informed methods have been proposed, for example, through inclusion of the current reconstructed PET image, alongside the MR image, in the iterative reconstruction process. In this present work, a number of kernel and maximum a posteriori (MAP) methodologies are compared, with the aim of identifying methods that enable a favorable trade-off between the suppression of noise and the retention of unique features present in the PET data.

METHODS

The reconstruction methods investigated were: the MR-informed conventional and spatially compact kernel methods, referred to as KEM and KEM largest value sparsification (LVS) respectively; the MR-informed Bowsher and Gaussian MR-guided MAP methods; and the PET-MR-informed hybrid kernel and anato-functional MAP methods. The trade-off between improving the reconstruction of the whole brain region and the PET-unique regions was investigated for all methods in comparison with postsmoothed maximum likelihood expectation maximization (MLEM), evaluated in terms of structural similarity index (SSIM), normalized root mean square error (NRMSE), bias, and standard deviation. Both simulated BrainWeb (10 noise realizations) and real [18 F] fluorodeoxyglucose (FDG) three-dimensional datasets were used. The real [18 F]FDG dataset was augmented with simulated tumors to allow comparison of the reconstruction methodologies for the case of known regions of PET-MR discrepancy and evaluated at full counts (100%) and at a reduced (10%) count level.

RESULTS

For the high-count simulated and real data studies, the anato-functional MAP method performed better than the other methods under investigation (MR-informed, PET-MR-informed and postsmoothed MLEM), in terms of achieving the best trade-off for the reconstruction of the whole brain and PET-unique regions, assessed in terms of the SSIM, NRMSE, and bias vs standard deviation. The inclusion of PET information in the anato-functional MAP method enables the reconstruction of PET-unique regions to attain similarly low levels of bias as unsmoothed MLEM, while moderately improving the whole brain image quality for low levels of regularization. However, for low count simulated datasets the anato-functional MAP method performs poorly, due to the inclusion of noisy PET information in the regularization term. For the low counts simulated dataset, KEM LVS and to a lesser extent, HKEM performed better than the other methods under investigation in terms of achieving the best trade-off for the reconstruction of the whole brain and PET-unique regions, assessed in terms of the SSIM, NRMSE, and bias vs standard deviation.

CONCLUSION

For the reconstruction of noisy data, multiple MR-informed methods produce favorable whole brain vs PET-unique region trade-off in terms of the image quality metrics of SSIM and NRMSE, comfortably outperforming the whole image denoising of postsmoothed MLEM.

Networks for Nonlinear Diffusion Problems in Imaging

A multitude of imaging and vision tasks have seen recently a major transformation by deep learning methods and in particular by the application of convolutional neural networks. These methods achieve impressive results, even for applications where it is not apparent that convolutions are suited to capture the underlying physics. In this work, we develop a network architecture based on nonlinear diffusion processes, named DiffNet. By design, we obtain a nonlinear network architecture that is well suited for diffusion-related problems in imaging. Furthermore, the performed updates are explicit, by which we obtain better interpretability and generalisability compared to classical convolutional neural network architectures. The performance of DiffNet is tested on the inverse problem of nonlinear diffusion with the Perona-Malik filter on the STL-10 image dataset. We obtain competitive results to the established U-Net architecture, with a fraction of parameters and necessary training data.

Full-Vector Gradient for Multi-spectral or Multivariate Images

Gradient extraction is important for a lot of metrological applications such as Control Quality by Vision. In this work, we propose a full-vector gradient for multi-spectral sensors. The full-vector gradient extends Di Zenzo expression to take into account the non-orthogonality of the acquisition channels thanks to a Gram matrix. This expression is generic and independent from channel count. Results are provided for a color and a multi-spectral snapshot sensor. Then, we show the accuracy improvement of the gradient calculation by creating a dedicated objective test and from real images.

Evaluation of Parallel Level Sets and Bowsher's Method as Segmentation-Free Anatomical Priors for Time-of-Flight PET Reconstruction

In this article, we evaluate Parallel Level Sets (PLS) and Bowsher's method as segmentation-free anatomical priors for regularized brain positron emission tomography (PET) reconstruction. We derive the proximity operators for two PLS priors and use the EM-TV algorithm in combination with the first order primal-dual algorithm by Chambolle and Pock to solve the non-smooth optimization problem for PET reconstruction with PLS regularization. In addition, we compare the performance of two PLS versions against the symmetric and asymmetric Bowsher priors with quadratic and relative difference penalty function. For this aim, we first evaluate reconstructions of 30 noise realizations of simulated PET data derived from a real hybrid positron emission tomography/magnetic resonance imaging (PET/MR) acquisition in terms of regional bias and noise. Second, we evaluate reconstructions of a real brain PET/MR data set acquired on a GE Signa time-of-flight PET/MR in a similar way. The reconstructions of simulated and real 3D PET/MR data show that all priors were superior to post-smoothed maximum likelihood expectation maximization with ordered subsets (OSEM) in terms of bias-noise characteristics in different regions of interest where the PET uptake follows anatomical boundaries. Our implementation of the asymmetric Bowsher prior showed slightly superior performance compared with the two versions of PLS and the symmetric Bowsher prior. At very high regularization weights, all investigated anatomical priors suffer from the transfer of non-shared gradients.

Joint MR-PET Reconstruction Using a Multi-Channel Image Regularizer

While current state of the art MR-PET scanners enable simultaneous MR and PET measurements, the acquired data sets are still usually reconstructed separately. We propose a new multi-modality reconstruction framework using second order Total Generalized Variation (TGV) as a dedicated multi-channel regularization functional that jointly reconstructs images from both modalities. In this way, information about the underlying anatomy is shared during the image reconstruction process while unique differences are preserved. Results from numerical simulations and in-vivo experiments using a range of accelerated MR acquisitions and different MR image contrasts demonstrate improved PET image quality, resolution, and quantitative accuracy.

PET Reconstruction With an Anatomical MRI Prior Using Parallel Level Sets

The combination of positron emission tomography (PET) and magnetic resonance imaging (MRI) offers unique possibilities. In this paper we aim to exploit the high spatial resolution of MRI to enhance the reconstruction of simultaneously acquired PET data. We propose a new prior to incorporate structural side information into a maximum a posteriori reconstruction. The new prior combines the strengths of previously proposed priors for the same problem: it is very efficient in guiding the reconstruction at edges available from the side information and it reduces locally to edge-preserving total variation in the degenerate case when no structural information is available. In addition, this prior is segmentation-free, convex and no a priori assumptions are made on the correlation of edge directions of the PET and MRI images. We present results for a simulated brain phantom and for real data acquired by the Siemens Biograph mMR for a hardware phantom and a clinical scan. The results from simulations show that the new prior has a better trade-off between enhancing common anatomical boundaries and preserving unique features than several other priors. Moreover, it has a better mean absolute bias-to-mean standard deviation trade-off and yields reconstructions with superior relative l²-error and structural similarity index. These findings are underpinned by the real data results from a hardware phantom and a clinical patient confirming that the new prior is capable of promoting well-defined anatomical boundaries.

Joint reconstruction of multi-channel, spectral CT data via constrained total nuclear variation minimization

We explore the use of the recently proposed 'total nuclear variation' (TVN) as a regularizer for reconstructing multi-channel, spectral CT images. This convex penalty is a natural extension of the total variation (TV) to vector-valued images and has the advantage of encouraging common edge locations and a shared gradient direction among image channels. We show how it can be incorporated into a general, data-constrained reconstruction framework and derive update equations based on the first-order, primal-dual algorithm of Chambolle and Pock. Early simulation studies based on the numerical XCAT phantom indicate that the inter-channel coupling introduced by the TVN leads to better preservation of image features at high levels of regularization, compared to independent, channel-by-channel TV reconstructions.

Multimodal Image Reconstruction Using Supplementary Structural Information in Total Variation Regularization

In this paper, we propose an iterative reconstruction algorithm which uses available information from one dataset collected using one modality to increase the resolution and signal-to-noise ratio of one collected by another modality. The method operates on the structural information only which increases its suitability across various applications. Consequently, the main aim of this method is to exploit available supplementary data within the regularization framework. The source of primary and supplementary datasets can be acquired using complementary imaging modes where different types of information are obtained (e.g. in medical imaging: anatomical and functional). It is shown by extracting structural information from the supplementary image (direction of level sets) one can enhance the resolution of the other image. Notably, the method enhances edges that are common to both images while not suppressing features that show high contrast in the primary image alone. In our iterative algorithm we use available structural information within a modified total variation penalty term. We provide numerical experiments to show the advantages and feasibility of the proposed technique in comparison to other methods.