Blind image restoration by anisotropic regularization

A Fast Image Deformity Correction Algorithm for Underwater Turbulent Image Distortion

An algorithm correcting distortion based on estimating the pixel shift is proposed for the degradation caused by underwater turbulence. The distorted image is restored and reconstructed by reference frame selection and two-dimensional pixel registration. A support vector machine-based kernel correlation filtering algorithm is proposed and applied to improve the speed and efficiency of the correction algorithm. In order to validate the algorithm, laboratory experiments on a controlled simulation system of turbulent water and field experiments in rivers and oceans are carried out, and the experimental results are compared with traditional, theoretical model-based and particle image velocimetry-based restoration and reconstruction algorithms. Using subjective visual evaluation, image distortion has been effectively suppressed; based on an objective performance statistical analysis, the measured values are better than the traditional and formerly studied restoration and reconstruction algorithms. The method proposed in this paper is also much faster than the other algorithms. It can be concluded that the proposed algorithm can effectively improve the de-distortion effect of the underwater turbulence degraded image, and provide potential techniques for the accurate operation of underwater target detection in real time.

Blind Super Resolution of Real-Life Video Sequences

Super resolution (SR) for real-life video sequences is a challenging problem due to complex nature of the motion fields. In this paper, a novel blind SR method is proposed to improve the spatial resolution of video sequences, while the overall point spread function of the imaging system, motion fields, and noise statistics are unknown. To estimate the blur(s), first, a nonuniform interpolation SR method is utilized to upsample the frames, and then, the blur(s) is(are) estimated through a multi-scale process. The blur estimation process is initially performed on a few emphasized edges and gradually on more edges as the iterations continue. Also for faster convergence, the blur is estimated in the filter domain rather than the pixel domain. The high-resolution frames are estimated using a cost function that has the fidelity and regularization terms of type Huber-Markov random field to preserve edges and fine details. The fidelity term is adaptively weighted at each iteration using a masking operation to suppress artifacts due to inaccurate motions. Very promising results are obtained for real-life videos containing detailed structures, complex motions, fast-moving objects, deformable regions, or severe brightness changes. The proposed method outperforms the state of the art in all performed experiments through both subjective and objective evaluations. The results are available online at http://lyle.smu.edu/~rajand/Video_SR/.

Unified blind method for multi-image super-resolution and single/multi-image blur deconvolution

This paper presents, for the first time, a unified blind method for multi-image super-resolution (MISR or SR), single-image blur deconvolution (SIBD), and multi-image blur deconvolution (MIBD) of low-resolution (LR) images degraded by linear space-invariant (LSI) blur, aliasing, and additive white Gaussian noise (AWGN). The proposed approach is based on alternating minimization (AM) of a new cost function with respect to the unknown high-resolution (HR) image and blurs. The regularization term for the HR image is based upon the Huber-Markov random field (HMRF) model, which is a type of variational integral that exploits the piecewise smooth nature of the HR image. The blur estimation process is supported by an edge-emphasizing smoothing operation, which improves the quality of blur estimates by enhancing strong soft edges toward step edges, while filtering out weak structures. The parameters are updated gradually so that the number of salient edges used for blur estimation increases at each iteration. For better performance, the blur estimation is done in the filter domain rather than the pixel domain, i.e., using the gradients of the LR and HR images. The regularization term for the blur is Gaussian (L2 norm), which allows for fast noniterative optimization in the frequency domain. We accelerate the processing time of SR reconstruction by separating the upsampling and registration processes from the optimization procedure. Simulation results on both synthetic and real-life images (from a novel computational imager) confirm the robustness and effectiveness of the proposed method.

Image deconvolution with multi-stage convex relaxation and its perceptual evaluation

This paper proposes a new image deconvolution method using multi-stage convex relaxation, and presents a metric for perceptual evaluation of deconvolution results. Recent work in image deconvolution addresses the deconvolution problem via minimization with non-convex regularization. Since all regularization terms in the objective function are non-convex, this problem can be well modeled and solved by multi-stage convex relaxation. This method, adopted from machine learning, iteratively refines the convex relaxation formulation using concave duality. The newly proposed deconvolution method has outstanding performance in noise removal and artifact control. A new metric, transduced contrast-to-distortion ratio (TCDR), is proposed based on a human vision system (HVS) model that simulates human responses to visual contrasts. It is sensitive to ringing and boundary artifacts, and very efficient to compute. We conduct comprehensive perceptual evaluation of image deconvolution using visual signal-to-noise ratio (VSNR) and TCDR. Experimental results of both synthetic and real data demonstrate that our method indeed improves the visual quality of deconvolution results with low distortions and artifacts.

A well-posed multiscale regularization scheme for digital image denoising

We propose an edge adaptive digital image denoising and restoration scheme based on space dependent regularization. Traditional gradient based schemes use an edge map computed from gradients alone to drive the regularization. This may lead to the oversmoothing of the input image, and noise along edges can be amplified. To avoid these drawbacks, we make use of a multiscale descriptor given by a contextual edge detector obtained from local variances. Using a smooth transition from the computed edges, the proposed scheme removes noise in flat regions and preserves edges without oscillations. By incorporating a space dependent adaptive regularization parameter, image smoothing is driven along probable edges and not across them. The well-posedness of the corresponding minimization problem is proved in the space of functions of bounded variation. The corresponding gradient descent scheme is implemented and further numerical results illustrate the advantages of using the adaptive parameter in the regularization scheme. Compared with similar edge preserving regularization schemes, the proposed adaptive weight based scheme provides a better multiscale edge map, which in turn produces better restoration.

Blind deconvolution using generalized cross-validation approach to regularization parameter estimation

In this paper, we propose and present an algorithm for total variation (TV)-based blind deconvolution. Both the unknown image and blur can be estimated within an alternating minimization framework. With the generalized cross-validation (GCV) method, the regularization parameters associated with the unknown image and blur can be updated in alternating minimization steps. Experimental results confirm that the performance of the proposed algorithm is better than variational Bayesian blind deconvolution algorithms with Student's-t priors or a total variation prior.

No-reference blur assessment of digital pictures based on multifeature classifiers

In this paper, we address the problem of no-reference quality assessment for digital pictures corrupted with blur. We start with the generation of a large real image database containing pictures taken by human users in a variety of situations, and the conduction of subjective tests to generate the ground truth associated to those images. Based upon this ground truth, we select a number of high quality pictures and artificially degrade them with different intensities of simulated blur (gaussian and linear motion), totalling 6000 simulated blur images. We extensively evaluate the performance of state-of-the-art strategies for no-reference blur quantification in different blurring scenarios, and propose a paradigm for blur evaluation in which an effective method is pursued by combining several metrics and low-level image features. We test this paradigm by designing a no-reference quality assessment algorithm for blurred images which combines different metrics in a classifier based upon a neural network structure. Experimental results show that this leads to an improved performance that better reflects the images' ground truth. Finally, based upon the real image database, we show that the proposed method also outperforms other algorithms and metrics in realistic blur scenarios.

Blind image deconvolution using machine learning for three-dimensional microscopy

In this work, we propose a novel method for the regularization of blind deconvolution algorithms. The proposed method employs example-based machine learning techniques for modeling the space of point spread functions. During an iterative blind deconvolution process, a prior term attracts the point spread function estimates to the learned point spread function space. We demonstrate the usage of this regularizer within a Bayesian blind deconvolution framework and also integrate into the latter a method for noise reduction, thus creating a complete blind deconvolution method. The application of the proposed algorithm is demonstrated on synthetic and real-world three-dimensional images acquired by a wide-field fluorescence microscope, where the need for blind deconvolution algorithms is indispensable, yielding excellent results.

Backscatter-contour-attenuation joint estimation model for attenuation compensation in ultrasound imagery

Ultrasound B-scan exhibits shadowing and enhancement artifacts due to acoustic wave propagation and spatially varying scatter attenuation across layers of tissues. These artifacts hide underlying echo signals that are truly clinically indicative of diseases. Attenuation compensation estimates and corrects for shadowing and enhancement artifacts, which improves the quality of ultrasound imaging. Block-based attenuation compensation methods, widely employed in commercial scanners, produce results with resolutions limited by the block size. To obtain higher spatial resolution (as desired for quantitative analysis), we present a backscatter-contour-attenuation (BCA) joint estimation model for attenuation compensation in pulse-echo imaging using a set of self-consistent partial differential equations and a contour evolution model. The problem is posed as reconstructing sources of information from observations. We derive the joint estimation model from minimizing a cost functional of separated attributes with region-based isotropic regularizations. A three-step alternating minimization method is adopted towards a tractable numerical solution. Detailed numerical methods are described. The efficacy of the proposed approach is demonstrated using simulated and real images.

Fidelity optimization for aberration-tolerant hybrid imaging systems

Several phase-modulation functions have been reported to decrease the aberration variance of the modulation-transfer-function (MTF) in aberration-tolerant hybrid imaging systems. The choice of this phase-modulation function is crucial for optimization of the overall system performance. To prevent a significant loss in signal-to-noise ratio, it is common to enforce restorability constraints on the MTF, requiring trade of aberration-tolerance and noise-gain. Instead of optimizing specific MTF characteristics, we directly minimize the expected imaging-error of the joint design. This method is used to compare commonly used phase-modulation functions: the antisymmetric generalized cubic polynomial and fourth-degree rotational symmetric phase-modulation. The analysis shows how optimal imaging performance is obtained using moderate phase-modulation, and more importantly, the relative merits of the above functions.

Blind and semi-blind deblurring of natural images

A method for blind image deblurring is presented. The method only makes weak assumptions about the blurring filter and is able to undo a wide variety of blurring degradations. To overcome the ill-posedness of the blind image deblurring problem, the method includes a learning technique which initially focuses on the main edges of the image and gradually takes details into account. A new image prior, which includes a new edge detector, is used. The method is able to handle unconstrained blurs, but also allows the use of constraints or of prior information on the blurring filter, as well as the use of filters defined in a parametric manner. Furthermore, it works in both single-frame and multiframe scenarios. The use of constrained blur models appropriate to the problem at hand, and/or of multiframe scenarios, generally improves the deblurring results. Tests performed on monochrome and color images, with various synthetic and real-life degradations, without and with noise, in single-frame and multiframe scenarios, showed good results, both in subjective terms and in terms of the increase of signal to noise ratio (ISNR) measure. In comparisons with other state of the art methods, our method yields better results, and shows to be applicable to a much wider range of blurs.

Variational Bayesian sparse kernel-based blind image deconvolution with Student's-t priors

In this paper, we present a new Bayesian model for the blind image deconvolution (BID) problem. The main novelty of this model is the use of a sparse kernel-based model for the point spread function (PSF) that allows estimation of both PSF shape and support. In the herein proposed approach, a robust model of the BID errors and an image prior that preserves edges of the reconstructed image are also used. Sparseness, robustness, and preservation of edges are achieved by using priors that are based on the Student's-t probability density function (PDF). This pdf, in addition to having heavy tails, is closely related to the Gaussian and, thus, yields tractable inference algorithms. The approximate variational inference methodology is used to solve the corresponding Bayesian model. Numerical experiments are presented that compare this BID methodology to previous ones using both simulated and real data.

Variational bayesian blind deconvolution using a total variation prior

In this paper, we present novel algorithms for total variation (TV) based blind deconvolution and parameter estimation utilizing a variational framework. Using a hierarchical Bayesian model, the unknown image, blur, and hyperparameters for the image, blur, and noise priors are estimated simultaneously. A variational inference approach is utilized so that approximations of the posterior distributions of the unknowns are obtained, thus providing a measure of the uncertainty of the estimates. Experimental results demonstrate that the proposed approaches provide higher restoration performance than non-TV-based methods without any assumptions about the unknown hyperparameters.

Space-variant restoration of images degraded by camera motion blur

We examine the problem of restoration from multiple images degraded by camera motion blur. We consider scenes with significant depth variations resulting in space-variant blur. The proposed algorithm can be applied if the camera moves along an arbitrary curve parallel to the image plane, without any rotations. The knowledge of camera trajectory and camera parameters is not necessary. At the input, the user selects a region where depth variations are negligible. The algorithm belongs to the group of variational methods that estimate simultaneously a sharp image and a depth map, based on the minimization of a cost functional. To initialize the minimization, it uses an auxiliary window-based depth estimation algorithm. Feasibility of the algorithm is demonstrated by three experiments with real images.

A new and general method for blind shift-variant deconvolution of biomedical images

We present a new method for blind deconvolution of multiple noisy images blurred by a shift-variant point-spread-function (PSF). We focus on a setting in which several images of the same object are available, and a transformation between these images is known. This setting occurs frequently in biomedical imaging, for example in microscopy or in medical ultrasound imaging. By using the information from multiple observations, we are able to improve the quality of images blurred by a shift-variant filter, without prior knowledge of this filter. Also, in contrast to other work on blind and shift-variant deconvolution, in our approach no parametrization of the PSF is required. We evaluate the proposed method quantitatively on synthetically degraded data as well as qualitatively on 3D ultrasound images of liver. The algorithm yields good restoration results and proves to be robust even in presence of high noise levels in the images.

A primal-dual active-set method for non-negativity constrained total variation deblurring problems

This paper studies image deblurring problems using a total variation-based model, with a non-negativity constraint. The addition of the non-negativity constraint improves the quality of the solutions, but makes the solution process a difficult one. The contribution of our work is a fast and robust numerical algorithm to solve the non-negatively constrained problem. To overcome the nondifferentiability of the total variation norm, we formulate the constrained deblurring problem as a primal-dual program which is a variant of the formulation proposed by Chan, Golub, and Mulet for unconstrained problems. Here, dual refers to a combination of the Lagrangian and Fenchel duals. To solve the constrained primal-dual program, we use a semi-smooth Newton's method. We exploit the relationship between the semi-smooth Newton's method and the primal-dual active set method to achieve considerable simplification of the computations. The main advantages of our proposed scheme are: no parameters need significant adjustment, a standard inverse preconditioner works very well, quadratic rate of local convergence (theoretical and numerical), numerical evidence of global convergence, and high accuracy of solving the optimality system. The scheme shows robustness of performance over a wide range of parameters. A comprehensive set of numerical comparisons are provided against other methods to solve the same problem which show the speed and accuracy advantages of our scheme.

A unified approach to superresolution and multichannel blind deconvolution

This paper presents a new approach to the blind deconvolution and superresolution problem of multiple degraded low-resolution frames of the original scene. We do not assume any prior information about the shape of degradation blurs. The proposed approach consists of building a regularized energy function and minimizing it with respect to the original image and blurs, where regularization is carried out in both the image and blur domains. The image regularization based on variational principles maintains stable performance under severe noise corruption. The blur regularization guarantees consistency of the solution by exploiting differences among the acquired low-resolution images. Several experiments on synthetic and real data illustrate the robustness and utilization of the proposed technique in real applications.

Blind deconvolution using a variational approach to parameter, image, and blur estimation

Following the hierarchical Bayesian framework for blind deconvolution problems, in this paper, we propose the use of simultaneous autoregressions as prior distributions for both the image and blur, and gamma distributions for the unknown parameters (hyperparameters) of the priors and the image formation noise. We show how the gamma distributions on the unknown hyperparameters can be used to prevent the proposed blind deconvolution method from converging to undesirable image and blur estimates and also how these distributions can be inferred in realistic situations. We apply variational methods to approximate the posterior probability of the unknown image, blur, and hyperparameters and propose two different approximations of the posterior distribution. One of these approximations coincides with a classical blind deconvolution method. The proposed algorithms are tested experimentally and compared with existing blind deconvolution methods.

Multichannel blind deconvolution of spatially misaligned images

Existing multichannel blind restoration techniques assume perfect spatial alignment of channels, correct estimation of blur size, and are prone to noise. We developed an alternating minimization scheme based on a maximum a posteriori estimation with a priori distribution of blurs derived from the multichannel framework and a priori distribution of original images defined by the variational integral. This stochastic approach enables us to recover the blurs and the original image from channels severely corrupted by noise. We observe that the exact knowledge of the blur size is not necessary, and we prove that translation misregistration up to a certain extent can be automatically removed in the restoration process.

A soft double regularization approach to parametric blind image deconvolution

This paper proposes a blind image deconvolution scheme based on soft integration of parametric blur structures. Conventional blind image deconvolution methods encounter a difficult dilemma of either imposing stringent and inflexible preconditions on the problem formulation or experiencing poor restoration results due to lack of information. This paper attempts to address this issue by assessing the relevance of parametric blur information, and incorporating the knowledge into the parametric double regularization (PDR) scheme. The PDR method assumes that the actual blur satisfies up to a certain degree of parametric structure, as there are many well-known parametric blurs in practical applications. Further, it can be tailored flexibly to include other blur types if some prior parametric knowledge of the blur is available. A manifold soft parametric modeling technique is proposed to generate the blur manifolds, and estimate the fuzzy blur structure. The PDR scheme involves the development of the meaningful cost function, the estimation of blur support and structure, and the optimization of the cost function. Experimental results show that it is effective in restoring degraded images under different environments.

Blind deblurring of spiral CT images

To discriminate fine anatomical features in the inner ear, it has been desirable that spiral computed tomography (CT) may perform beyond their current resolution limits with the aid of digital image processing techniques. In this paper, we develop a blind deblurring approach to enhance image resolution retrospectively without complete knowledge of the underlying point spread function (PSF). An oblique CT image can be approximated as the convolution of an isotropic Gaussian PSF and the actual cross section. Practically, the parameter of the PSF is often unavailable. Hence, estimation of the parameter for the underlying PSF is crucially important for blind image deblurring. Based on the iterative deblurring theory, we formulate an edge-to-noise ratio (ENR) to characterize the image quality change due to deblurring. Our blind deblurring algorithm estimates the parameter of the PSF by maximizing the ENR, and deblurs images. In the phantom studies, the blind deblurring algorithm reduces image blurring by about 24%, according to our blurring residual measure. Also, the blind deblurring algorithm works well in patient studies. After fully automatic blind deblurring, the conspicuity of the submillimeter features of the cochlea is substantially improved.