Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution

O-PRESS: Boosting OCT axial resolution with Prior guidance, Recurrence, and Equivariant Self-Supervision

Optical coherence tomography (OCT) is a noninvasive technology that enables real-time imaging of tissue microanatomies. The axial resolution of OCT is intrinsically constrained by the spectral bandwidth of the employed light source while maintaining a fixed center wavelength for a specific application. Physically extending this bandwidth faces strong limitations and requires a substantial cost. We present a novel computational approach, called as O-PRESS, for boosting the axial resolution of OCT with Prior guidance, a Recurrent mechanism, and Equivariant Self-Supervision. Diverging from conventional deconvolution methods that rely on physical models or data-driven techniques, our method seamlessly integrates OCT modeling and deep learning, enabling us to achieve real-time axial-resolution enhancement exclusively from measurements without a need for paired images. Our approach solves two primary tasks of resolution enhancement and noise reduction with one treatment. Both tasks are executed in a self-supervised manner, with equivariance imaging and free space priors guiding their respective processes. Experimental evaluations, encompassing both quantitative metrics and visual assessments, consistently verify the efficacy and superiority of our approach, which exhibits performance on par with fully supervised methods. Importantly, the robustness of our model is affirmed, showcasing its dual capability to enhance axial resolution while concurrently improving the signal-to-noise ratio.

Blind multi-Poissonian image deconvolution with sparse log-step gradient prior

Blind image deconvolution plays a very important role in the fields such as astronomical observation and fluorescence microscopy imaging, in which the noise follows Poisson distribution. However, due to the ill-posedness, it is a very challenging task to reach a satisfactory result from a single blurred image especially when the power of the Poisson noise is at a high level. Therefore, in this paper, we try to achieve high-quality restoration results with multi-blurred images which are contaminated by Poisson noise. Firstly, we design a novel sparse log-step gradient prior which adopts a mixture of logarithm and step functions to regularize the image gradients and combine it with the Poisson distribution to formulate the blind multi-image deconvolution problem. Secondly, we incorporate the methods of variable splitting and Lagrange multiplier to convert the original problem into sub-problems, then we alternately solve them to achieve the estimation of all the blur kernels of corresponding blurred images. Besides, we also design a non-blind multi-image deconvolution algorithm which is based on the log-step gradient prior to reach the final restored image. Experimental results on both synthetic and real-world blurred images show that the proposed prior is very capable of suppressing negative artifacts caused by ill-posedness. The algorithm can achieve restored image of very high quality which is competitive with some state-of-the-art methods.

Video Satellite Imagery Super-Resolution via Model-Based Deep Neural Networks

Video satellite imagery has become a hot research topic in Earth observation due to its ability to capture dynamic information. However, its high temporal resolution comes at the expense of spatial resolution. In recent years, deep learning (DL) based super-resolution (SR) methods have played an essential role to improve the spatial resolution of video satellite images. Instead of fully considering the degradation process, most existing DL-based methods attempt to learn the relationship between low-resolution (LR) satellite video frames and their corresponding high-resolution (HR) ones. In this paper, we propose model-based deep neural networks for video satellite imagery SR (VSSR). The VSSR is composed of three main modules: degradation estimation module, intermediate image generation module, and multi-frame feature fusion module. First, the blur kernel and noise level of LR video frames are flexibly estimated by the degradation estimation module. Second, an intermediate image generation module is proposed to iteratively solve two optimal subproblems and the outputs of this module are intermediate SR frames. Third, a three-dimensional (3D) feature fusion subnetwork is leveraged to fuse the features from multiple video frames. Different from previous video satellite SR methods, the proposed VSSR is a multi-frame-based method that can merge the advantages of both learning-based and model-based methods. Experiments on real-world Jilin-1 and OVS-1 video satellite images have been conducted and the SR results demonstrate that the proposed VSSR achieves superior visual effects and quantitative performance compared with the state-of-the-art methods.

Low-dose spectral CT reconstruction using image gradient ℓ 0-norm and tensor dictionary

Spectral computed tomography (CT) has a great superiority in lesion detection, tissue characterization and material decomposition. To further extend its potential clinical applications, in this work, we propose an improved tensor dictionary learning method for low-dose spectral CT reconstruction with a constraint of image gradient ℓ ₀-norm, which is named as ℓ ₀TDL. The ℓ ₀TDL method inherits the advantages of tensor dictionary learning (TDL) by employing the similarity of spectral CT images. On the other hand, by introducing the ℓ ₀-norm constraint in gradient image domain, the proposed method emphasizes the spatial sparsity to overcome the weakness of TDL on preserving edge information. The split-bregman method is employed to solve the proposed method. Both numerical simulations and real mouse studies are perform to evaluate the proposed method. The results show that the proposed ℓ ₀TDL method outperforms other competing methods, such as total variation (TV) minimization, TV with low rank (TV+LR), and TDL methods.

Learning-based subject-specific estimation of dynamic maps of cortical morphology at missing time points in longitudinal infant studies

Longitudinal neuroimaging analysis of the dynamic brain development in infants has received increasing attention recently. Many studies expect a complete longitudinal dataset in order to accurately chart the brain developmental trajectories. However, in practice, a large portion of subjects in longitudinal studies often have missing data at certain time points, due to various reasons such as the absence of scan or poor image quality. To make better use of these incomplete longitudinal data, in this paper, we propose a novel machine learning-based method to estimate the subject-specific, vertex-wise cortical morphological attributes at the missing time points in longitudinal infant studies. Specifically, we develop a customized regression forest, named dynamically assembled regression forest (DARF), as the core regression tool. DARF ensures the spatial smoothness of the estimated maps for vertex-wise cortical morphological attributes and also greatly reduces the computational cost. By employing a pairwise estimation followed by a joint refinement, our method is able to fully exploit the available information from both subjects with complete scans and subjects with missing scans for estimation of the missing cortical attribute maps. The proposed method has been applied to estimating the dynamic cortical thickness maps at missing time points in an incomplete longitudinal infant dataset, which includes 31 healthy infant subjects, each having up to five time points in the first postnatal year. The experimental results indicate that our proposed framework can accurately estimate the subject-specific vertex-wise cortical thickness maps at missing time points, with the average error less than 0.23 mm. Hum Brain Mapp 37:4129-4147, 2016. © 2016 Wiley Periodicals, Inc.

Variational Bayesian Approach to Multiframe Image Restoration

Image restoration is a fundamental problem in the field of image processing. The key objective of image restoration is to recover clean images from images degraded by noise and blur. Recently, a family of new statistical techniques called variational Bayes (VB) has been introduced to image restoration, which enables us to automatically tune parameters that control restoration. While information from one image is often insufficient for high-quality restoration, however, current state-of-the-art methods of image restoration via VB approaches use only a single-degraded image to recover a clean image. In this paper, we propose a novel method of multiframe image restoration via a VB approach, which can achieve higher image quality while tuning parameters automatically. Given multiple degraded images, this method jointly estimates a clean image and other parameters, including an image warping parameter introduced for the use of multiple images, through Bayesian inference that we enable by making full use of VB techniques. Through various experiments, we demonstrate the effectiveness of our multiframe method by comparing it with single-frame one, and also show the advantages of our VB approach over non-VB approaches.

Multi-Image Deblurring Using Complementary Sets of Fluttering Patterns

We present a novel coded exposure video technique for multi-image motion deblurring. The key idea of this paper is to capture video frames with a set of complementary fluttering patterns, which enables us to preserve all spectrum bands of a latent image and recover a sharp latent image. To achieve this, we introduce an algorithm for generating a complementary set of binary sequences based on the modern communication theory and implement the coded exposure video system with an off-the-shelf machine vision camera. To demonstrate the effectiveness of our method, we provide in-depth analyses of the theoretical bounds and the spectral gains of our method and other state-of-the-art computational imaging approaches. We further show deblurring results on various challenging examples with quantitative and qualitative comparisons to other computational image capturing methods used for image deblurring, and show how our method can be applied for protecting privacy in videos.

Edge-preserving reconstruction from sparse projections of limited-angle computed tomography using ℓ0-regularized gradient prior

Accurate images reconstructed from limited computed tomography (CT) data are desired when reducing the X-ray radiation exposure imposed on patients. The total variation (TV), known as the l₁-norm of the image gradient magnitudes, is popular in CT reconstruction from incomplete projection data. However, as the projection data collected are from a sparse-view of the limited scanning angular range, the results reconstructed by a TV-based method suffer from blocky artifact and gradual changed artifacts near the edges, which in turn make the reconstruction images degraded. Different from the TV, the ℓ₀-norm of an image gradient counts the number of its non-zero coefficients of the image gradient. Since the regularization based on the ℓ₀-norm of the image gradient will not penalize the large gradient magnitudes, the edge can be effectively retained. In this work, an edge-preserving image reconstruction method based on l₀-regularized gradient prior was investigated for limited-angle computed tomography from sparse projections. To solve the optimization model effectively, the variable splitting and the alternating direction method (ADM) were utilized. Experiments demonstrated that the ADM-like method used for the non-convex optimization problem has better performance than other classical iterative reconstruction algorithms in terms of edge preservation and artifact reduction.

Blind Super Resolution of Real-Life Video Sequences

Super resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall point spread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multi-scale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations. The results are available online at http://lyle.smu.edu/~rajand/Video_SR/.

ℓ0 Gradient Minimization Based Image Reconstruction for Limited-Angle Computed Tomography

In medical and industrial applications of computed tomography (CT) imaging, limited by the scanning environment and the risk of excessive X-ray radiation exposure imposed to the patients, reconstructing high quality CT images from limited projection data has become a hot topic. X-ray imaging in limited scanning angular range is an effective imaging modality to reduce the radiation dose to the patients. As the projection data available in this modality are incomplete, limited-angle CT image reconstruction is actually an ill-posed inverse problem. To solve the problem, image reconstructed by conventional filtered back projection (FBP) algorithm frequently results in conspicuous streak artifacts and gradual changed artifacts nearby edges. Image reconstruction based on total variation minimization (TVM) can significantly reduce streak artifacts in few-view CT, but it suffers from the gradual changed artifacts nearby edges in limited-angle CT. To suppress this kind of artifacts, we develop an image reconstruction algorithm based on ℓ₀ gradient minimization for limited-angle CT in this paper. The ℓ₀-norm of the image gradient is taken as the regularization function in the framework of developed reconstruction model. We transformed the optimization problem into a few optimization sub-problems and then, solved these sub-problems in the manner of alternating iteration. Numerical experiments are performed to validate the efficiency and the feasibility of the developed algorithm. From the statistical analysis results of the performance evaluations peak signal-to-noise ratio (PSNR) and normalized root mean square distance (NRMSD), it shows that there are significant statistical differences between different algorithms from different scanning angular ranges (p<0.0001). From the experimental results, it also indicates that the developed algorithm outperforms classical reconstruction algorithms in suppressing the streak artifacts and the gradual changed artifacts nearby edges simultaneously.