VIDOSAT: High-Dimensional Sparsifying Transform Learning for Online Video Denoising

When Laplacian Scale Mixture Meets Three-Layer Transform: A Parametric Tensor Sparsity for Tensor Completion

Recently, tensor sparsity modeling has achieved great success in the tensor completion (TC) problem. In real applications, the sparsity of a tensor can be rationally measured by low-rank tensor decomposition. However, existing methods either suffer from limited modeling power in estimating accurate rank or have difficulty in depicting hierarchical structure underlying such data ensembles. To address these issues, we propose a parametric tensor sparsity measure model, which encodes the sparsity for a general tensor by Laplacian scale mixture (LSM) modeling based on three-layer transform (TLT) for factor subspace prior with Tucker decomposition. Specifically, the sparsity of a tensor is first transformed into factor subspace, and then factor sparsity in the gradient domain is used to express the local similarity in within-mode. To further refine the sparsity, we adopt LSM by the transform learning scheme to self-adaptively depict deeper layer structured sparsity, in which the transformed sparse matrices in the sense of a statistical model can be modeled as the product of a Laplacian vector and a hidden positive scalar multiplier. We call the method as parametric tensor sparsity delivered by LSM-TLT. By a progressive transformation operator, we formulate the LSM-TLT model and use it to address the TC problem, and then the alternating direction method of multipliers-based optimization algorithm is designed to solve the problem. The experimental results on RGB images, hyperspectral images (HSIs), and videos demonstrate the proposed method outperforms state of the arts.

Multilayer residual sparsifying transform (MARS) model for low-dose CT image reconstruction

PURPOSE

Signal models based on sparse representations have received considerable attention in recent years. On the other hand, deep models consisting of a cascade of functional layers, commonly known as deep neural networks, have been highly successful for the task of object classification and have been recently introduced to image reconstruction. In this work, we develop a new image reconstruction approach based on a novel multilayer model learned in an unsupervised manner by combining both sparse representations and deep models. The proposed framework extends the classical sparsifying transform model for images to a Multilayer residual sparsifying transform (MARS) model, wherein the transform domain data are jointly sparsified over layers. We investigate the application of MARS models learned from limited regular-dose images for low-dose CT reconstruction using penalized weighted least squares (PWLS) optimization.

METHODS

We propose new formulations for multilayer transform learning and image reconstruction. We derive an efficient block coordinate descent algorithm to learn the transforms across layers, in an unsupervised manner from limited regular-dose images. The learned model is then incorporated into the low-dose image reconstruction phase.

RESULTS

Low-dose CT experimental results with both the XCAT phantom and Mayo Clinic data show that the MARS model outperforms conventional methods such as filtered back-projection and PWLS methods based on the edge-preserving (EP) regularizer in terms of two numerical metrics (RMSE and SSIM) and noise suppression. Compared with the single-layer learned transform (ST) model, the MARS model performs better in maintaining some subtle details.

CONCLUSIONS

This work presents a novel data-driven regularization framework for CT image reconstruction that exploits learned multilayer or cascaded residual sparsifying transforms. The image model is learned in an unsupervised manner from limited images. Our experimental results demonstrate the promising performance of the proposed multilayer scheme over single-layer learned sparsifying transforms. Learned MARS models also offer better image quality than typical nonadaptive PWLS methods.

Spatial-Spectral Structured Sparse Low-Rank Representation for Hyperspectral Image Super-Resolution

Hyperspectral image super-resolution by fusing high-resolution multispectral image (HR-MSI) and low-resolution hyperspectral image (LR-HSI) aims at reconstructing high resolution spatial-spectral information of the scene. Existing methods mostly based on spectral unmixing and sparse representation are often developed from a low-level vision task perspective, they cannot sufficiently make use of the spatial and spectral priors available from higher-level analysis. To this issue, this paper proposes a novel HSI super-resolution method that fully considers the spatial/spectral subspace low-rank relationships between available HR-MSI/LR-HSI and latent HSI. Specifically, it relies on a new subspace clustering method named "structured sparse low-rank representation" (SSLRR), to represent the data samples as linear combinations of the bases in a given dictionary, where the sparse structure is induced by low-rank factorization for the affinity matrix. Then we exploit the proposed SSLRR model to learn the SSLRR along spatial/spectral domain from the MSI/HSI inputs. By using the learned spatial and spectral low-rank structures, we formulate the proposed HSI super-resolution model as a variational optimization problem, which can be readily solved by the ADMM algorithm. Compared with state-of-the-art hyperspectral super-resolution methods, the proposed method shows better performance on three benchmark datasets in terms of both visual and quantitative evaluation.

Online Tensor Sparsifying Transform based on Temporal Superpixels from Compressive Spectral Video Measurements

Spectral videos contain highly redundant information across spatial, spectral and temporal axes which can be exploited through a temporal-data-learned sparsifying basis. However, in compressive spectral video acquisition, tackling dictionary learning is time-consuming since it increases the computational complexity and presents drawbacks for realtime processing, where offline learning is required. This paper introduces a tensor-decomposition learning (TenDL) framework for simultaneous online sparsifying and recovering the spatialspectral- temporal information of a spectral video performed on several temporal superpixels (TSP-TenDL) for time processing reduction. The framework is composed of two main stages: preprocessing and joint estimation. The preprocessing stage includes a strategy for a grayscale approximation of the video to provide a suitable initialization of the sparsifying basis to be learned. To fully exploit the high signal correlation, a set of temporal superpixels is estimated from the grayscale approximation, reducing the reconstruction time of the large-scale data. Then, the outcome of the first stage is used to estimate the basis and the signal coefficients, where an optimization problem is solved to learn and reconstruct the basis and the signal, respectively, following a block-descent coordinate strategy. The proposed approach is compared from simulations with an offline-learned based method, traditional matrix-based recovery algorithms and the tensor-based recovery, the two latter using a fixed basis, where TSP-TenDL exhibits higher image quality results and lower computation time. Specifically, our methodology gains up to 7dB in terms of PSNR and a speedup of up to 6.6× compared with state-of-the-art counterparts.

Online Adaptive Image Reconstruction (OnAIR) Using Dictionary Models

Sparsity and low-rank models have been popular for reconstructing images and videos from limited or corrupted measurements. Dictionary or transform learning methods are useful in applications such as denoising, inpainting, and medical image reconstruction. This paper proposes a framework for online (or time-sequential) adaptive reconstruction of dynamic image sequences from linear (typically undersampled) measurements. We model the spatiotemporal patches of the underlying dynamic image sequence as sparse in a dictionary, and we simultaneously estimate the dictionary and the images sequentially from streaming measurements. Multiple constraints on the adapted dictionary are also considered such as a unitary matrix, or low-rank dictionary atoms that provide additional efficiency or robustness. The proposed online algorithms are memory efficient and involve simple updates of the dictionary atoms, sparse coefficients, and images. Numerical experiments demonstrate the usefulness of the proposed methods in inverse problems such as video reconstruction or inpainting from noisy, subsampled pixels, and dynamic magnetic resonance image reconstruction from very limited measurements.

Motion Estimation-Assisted Denoising for an Efficient Combination with an HEVC Encoder

Noise, which is commonly generated in low-light environments or by low-performance cameras, is a major cause of the degradation of compression efficiency. In previous studies that attempted to combine a denoise algorithm and a video encoder, denoising was used independently of the code for pre-processing or post-processing. However, this process must be tightly coupled with encoding because noise affects the compression efficiency greatly. In addition, this represents a major opportunity to reduce the computational complexity, because the encoding process and a denoise algorithm have many similarities. In this paper, a simple, add-on denoising scheme is proposed through a combination of high-efficiency video coding (HEVC) and block matching three-dimensional collaborative filtering (BM3D) algorithms. It is known that BM3D has excellent denoise performance but that it is limited in its use due to its high computational complexity. This paper employs motion estimation in HEVC to replace the block matching of BM3D so that most of the time-consuming functions are shared. To overcome the challenging algorithmic differences, the hierarchical structure in HEVC is uniquely utilized. As a result, the computational complexity is drastically reduced while the competitive performance capabilities in terms of coding efficiency and denoising quality are maintained.